Building OpenOCD

1. System

I work on a VirtualBox 6.0 virtual machine running Ubuntu 19.10 desktop 64-bit on Windows 10, installed according to the following instructions:


I’m going to build OpenOCD based on the xPacks project from Liviu Ionescu:

The OpenOCD build instructions are documented here: https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/README-BUILD.md


2. Install Git

 $ sudo apt install git

 


3 The Ubuntu XBB

3.1 Install Ubuntu XBB

First I install the Ubuntu XBB (xPack Build Box), following the guide at https://github.com/xpack/xpack-build-box/tree/master/ubuntu#how-to-install

 $ rm -rf "${HOME}"/Downloads/xpack-build-box.git
 $ git clone --recurse-submodules https://github.com/xpack/xpack-build-box.git \
   "${HOME}"/Downloads/xpack-build-box.git
 $ sudo bash "${HOME}"/Downloads/xpack-build-box.git/ubuntu/install-xbb.sh
 $ sudo bash "${HOME}"/Downloads/xpack-build-box.git/ubuntu/add-xbb-extras.sh

3.2 Use Ubuntu XBB

Now XBB is installed. Here are some explanations from the guide on how to use XBB. I don’t insert these commands right now. I keep them in the back of my mind:

Using XBB

# At init time:
 $ source "/opt/xbb/xbb-source.sh"
# Extra verbosity:
 $ chmod +x /opt/xbb/bin/pkg-config-verbose
 $ export PKG_CONFIG=pkg-config-verbose
(
    # When needed; preferably in a sub-shell.
    $ xbb_activate

    $ ../configure
    $ make
)

And here is some more explanation from the guide:

About 'xbb-source.sh'

xbb-source.sh should be included with source by the build scripts, to define more bash functions to the shell. These functions are used to extend the environment with resources available in the XBB folders.

The xbb_activate function is used to extend the PATH and the LD_LIBRARY_PATH with folders in the XBB folders, in front of existing folders, so that the XBB executables are preferred over the system ones.

The xbb_activate_this function is used to further extend the environment with other definitions, like PKG_CONFIG_PATH , the path where pkg-config searches for resources, if it is necessary to search for the XBB folders. There are also custom variables that can be used as CPPFLAGS and LDFLAGS, that add the XBB folders to the include paths and the library path.

The 'pkg-config-verbose script'

While running the configuration step, it is sometimes useful to trace how pkg-config identifies resources to be used during the build. The standard pkg-config does not have an option to increase verbosity.

The workaround is to use a separate script that displays the received command and the response on the stderr stream. This script is not specific to XBB, and can be used with any build.

For this, copy the file into .../xbb/bin or any other folder present in the PATH and pass the script name via the environment:

 $ chmod +x /opt/xbb/bin/pkg-config-verbose
 $ export PKG_CONFIG=pkg-config-verbose

I believe there is no more need to copy the pkg-config-verbose file, as it is already there:

 $ cd /opt/xbb/bin/
 $ ls
 pkg-config-verbose

I will issue the chmod and export commands when I need the verbosity.

3.3 Ubuntu libraries

I issue the following command from the guide:

$ dpkg -l \
libpng-dev \
libjpeg-dev \
libsdl2-dev \
libsdl2-image-dev \
libpixman-1-dev \
libglib2.0-dev \
zlib1g-dev \
libffi-dev \
libxml2-dev \
zlib1g-dev \

The result I get is:

dpkg-query: no packages found matching libpng-dev
dpkg-query: no packages found matching libjpeg-dev
dpkg-query: no packages found matching libsdl2-dev
dpkg-query: no packages found matching libsdl2-image-dev
dpkg-query: no packages found matching libpixman-1-dev
dpkg-query: no packages found matching libglib2.0-dev
dpkg-query: no packages found matching zlib1g-dev
dpkg-query: no packages found matching libffi-dev
dpkg-query: no packages found matching libxml2-dev
dpkg-query: no packages found matching zlib1g-dev

While the guide shows:

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  libffi-dev:amd 3.2.1-8      amd64        Foreign Function Interface librar
ii  libglib2.0-dev 2.56.3-0ubun amd64        Development files for the GLib li
ii  libjpeg-dev:am 8c-2ubuntu8  amd64        Independent JPEG Group's JPEG run
ii  libpixman-1-de 0.34.0-2     amd64        pixel-manipulation library for X 
ii  libpng-dev:amd 1.6.34-1ubun amd64        PNG library - development (versio
ii  libsdl2-dev:am 2.0.8+dfsg1- amd64        Simple DirectMedia Layer developm
ii  libsdl2-image- 2.0.3+dfsg1- amd64        Image loading library for Simple 
ii  libxml2-dev:am 2.9.4+dfsg1- amd64        Development files for the GNOME X
ii  zlib1g-dev:amd 1:1.2.11.dfs amd64        compression library - development

So I decide to install them all:

 $ sudo apt-get install libpng-dev
 $ sudo apt-get install libjpeg-dev
 $ sudo apt-get install libsdl2-dev
 $ sudo apt-get install libsdl2-image-dev
 $ sudo apt-get install libpixman-1-dev
 $ sudo apt-get install libglib2.0-dev
 $ sudo apt-get install zlib1g-dev
 $ sudo apt-get install libffi-dev
 $ sudo apt-get install libxml2-dev
 $ sudo apt-get install zlib1g-dev

Now I get the following result:

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                    Version                Architecture Description
+++-=======================-======================-============-=======================================================================
ii  libffi-dev:amd64        3.2.1-9                amd64        Foreign Function Interface library (development files)
ii  libglib2.0-dev:amd64    2.62.1-1               amd64        Development files for the GLib library
ii  libjpeg-dev:amd64       8c-2ubuntu8            amd64        Independent JPEG Group's JPEG runtime library (dependency package)
ii  libpixman-1-dev:amd64   0.38.4-0ubuntu1        amd64        pixel-manipulation library for X and cairo (development files)
ii  libpng-dev:amd64        1.6.37-1               amd64        PNG library - development (version 1.6)
ii  libsdl2-dev             2.0.10+dfsg1-1ubuntu1  amd64        Simple DirectMedia Layer development files
ii  libsdl2-image-dev:amd64 2.0.5+dfsg1-1          amd64        Image loading library for Simple DirectMedia Layer 2, development files
ii  libxml2-dev:amd64       2.9.4+dfsg1-7ubuntu3   amd64        Development files for the GNOME XML library
ii  zlib1g-dev:amd64        1:1.2.11.dfsg-1ubuntu3 amd64        compression library - development

I believe my Ubuntu OS (and the XBB installed on it) is now ready to jump to the next guide.


4 Build OpenOCD

Installing OpenOCD is explained here: https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/README-BUILD.md

4.1 Pull openocd-xpack.git

I run the following commands:

 $ rm -rf ~/Downloads/openocd-xpack.git
 $ git clone --recurse-submodules --branch xpack-develop https://github.com/xpack-dev-tools/openocd-xpack.git \
~/Downloads/openocd-xpack.git

Note: I chose to set the --branch xpack-develop flag. Not sure if that’s wise

Note: The guide continues with How to build local/native binaries and How to build distributions. I ignore the first and continue with the second one.

4.2 Prerequisites

4.2.1 Install Docker

The guide jumps to the page https://xpack.github.io/xbb/prerequisites/
On that page is explained how to install docker and run CentOS 6 in it with XBB. This confuses me, because I just installed XBB natively on my Ubuntu. Should I still install docker? I will follow the guide anyhow.

$ sudo apt-get update
$ sudo apt-get install apt-transport-https ca-certificates  curl software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository  "deb [arch=amd64] https://download.docker.com/linux/ubuntu  $(lsb_release -cs)  stable"
$ sudo apt-get update
$ sudo apt-get -y install docker-ce

The procedure goes wrong at the fourth command. I get the following output:

sudo add-apt-repository  "deb [arch=amd64] https://download.docker.com/linux/ubuntu  $(lsb_release -cs)  stable"
Hit:1 http://be.archive.ubuntu.com/ubuntu eoan InRelease
Hit:2 http://be.archive.ubuntu.com/ubuntu eoan-updates InRelease                         
Hit:3 http://be.archive.ubuntu.com/ubuntu eoan-backports InRelease                                      
Get:4 http://security.ubuntu.com/ubuntu eoan-security InRelease [97,5 kB]                               
Ign:5 https://download.docker.com/linux/ubuntu eoan InRelease
Err:6 https://download.docker.com/linux/ubuntu eoan Release
  404  Not Found [IP: 13.225.38.15 443]
Reading package lists... Done                             
E: The repository 'https://download.docker.com/linux/ubuntu eoan Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

I apply the workaround mentioned by @Tommy Murphy (see https://github.com/docker/for-linux/issues/833#issuecomment-544236041):

wget "https://download.docker.com/linux/ubuntu/dists/disco/pool/stable/amd64/containerd.io_1.2.6-3_amd64.deb"
wget "https://download.docker.com/linux/ubuntu/dists/disco/pool/stable/amd64/docker-ce-cli_19.03.3~3-0~ubuntu-disco_amd64.deb"
wget "https://download.docker.com/linux/ubuntu/dists/disco/pool/stable/amd64/docker-ce_19.03.3~3-0~ubuntu-disco_amd64.deb"
sudo dpkg -i "containerd.io_1.2.6-3_amd64.deb"
sudo dpkg -i "docker-ce-cli_19.03.3~3-0~ubuntu-disco_amd64.deb"
sudo dpkg -i "docker-ce_19.03.3~3-0~ubuntu-disco_amd64.deb"

Then I issue the command:

 $ sudo apt-get -y install docker-ce
 Reading package lists... Done
 Building dependency tree       
 Reading state information... Done
 docker-ce is already the newest version (5:19.03.3~3-0~ubuntu-disco).
 0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

I believe Docker is installed now. I run:

 $ sudo docker run hello-world

Docker is correctly installed!

4.2.2 Configure Docker to run as regular user

I run these commands:

 $ sudo groupadd docker
 $ sudo gpasswd -a ${USER} docker
 $ sudo service docker restart

Now Docker should work without sudo command. To test, run:

 $ docker run hello-world

It works!

4.3 Build distributions

I navigate to the ~/Downloads/openocd-xpack.git folder, where the https://github.com/xpack-dev-tools/openocd-xpack.git repo was cloned:

 $ cd ~/Downloads/openocd-xpack.git

Now it’s time for the git commands:

 $ git checkout master
warning: unable to rmdir 'scripts/helper': Directory not empty
Branch 'master' set up to track remote branch 'master' from 'origin'.
Switched to a new branch 'master'

 $ git remote add upstream git://git.code.sf.net/p/openocd/code

 $ git remote -v
origin	https://github.com/xpack-dev-tools/openocd-xpack.git (fetch)
origin	https://github.com/xpack-dev-tools/openocd-xpack.git (push)
upstream	git://git.code.sf.net/p/openocd/code (fetch)
upstream	git://git.code.sf.net/p/openocd/code (push)

 $ git merge upstream/master
merge: upstream/master - not something we can merge

The merge failed, probably because Git must first pull the upstream in locally.

 $ git fetch upstream
warning: no common commits
remote: Enumerating objects: 62390, done.
remote: Counting objects: 100% (62390/62390), done.
[...]

# Checkout the upstream master.
# To avoid conflicts, Git doesn't overwrite the local
# master label, so the checked out upstream/master
# is without label (headless).
 $ git checkout upstream/master
Note: checking out 'upstream/master'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at f9809950 mips_ejtag: there is no DCR.MIPS64 bit

# Checkout the local master again.
 $ git checkout master
Previous HEAD position was f9809950 mips_ejtag: there is no DCR.MIPS64 bit
Switched to branch 'master'
Your branch is up to date with 'origin/master'.

Now I try to merge again:

 $ git merge upstream/master
fatal: refusing to merge unrelated histories

I start over again:

1. System

I work on a VirtualBox 6.0 virtual machine running Ubuntu 19.10 desktop 64-bit on Windows 10, installed according to the following instructions:


I’m going to build OpenOCD based on the xPacks project from Liviu Ionescu:

The OpenOCD build instructions are documented here: https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/README-BUILD.md


2. Install Git

 $ sudo apt install git

 


3. Prerequisites (Docker)

3.1 Install Docker

The guide jumps to the page https://xpack.github.io/xbb/prerequisites/
On that page is explained how to install docker and run CentOS 6 in it with XBB. This confuses me, because I just installed XBB natively on my Ubuntu. Should I still install docker? I will follow the guide anyhow.

$ sudo apt-get update
$ sudo apt-get install apt-transport-https ca-certificates  curl software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository  "deb [arch=amd64] https://download.docker.com/linux/ubuntu  $(lsb_release -cs)  stable"
$ sudo apt-get update
$ sudo apt-get -y install docker-ce

The procedure goes wrong at the fourth command. I get the following output:

sudo add-apt-repository  "deb [arch=amd64] https://download.docker.com/linux/ubuntu  $(lsb_release -cs)  stable"
Hit:1 http://be.archive.ubuntu.com/ubuntu eoan InRelease
Hit:2 http://be.archive.ubuntu.com/ubuntu eoan-updates InRelease                         
Hit:3 http://be.archive.ubuntu.com/ubuntu eoan-backports InRelease                                      
Get:4 http://security.ubuntu.com/ubuntu eoan-security InRelease [97,5 kB]                               
Ign:5 https://download.docker.com/linux/ubuntu eoan InRelease
Err:6 https://download.docker.com/linux/ubuntu eoan Release
  404  Not Found [IP: 13.225.38.15 443]
Reading package lists... Done                             
E: The repository 'https://download.docker.com/linux/ubuntu eoan Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

I apply the workaround mentioned by @Tommy Murphy (see https://github.com/docker/for-linux/issues/833#issuecomment-544236041):

wget "https://download.docker.com/linux/ubuntu/dists/disco/pool/stable/amd64/containerd.io_1.2.6-3_amd64.deb"
wget "https://download.docker.com/linux/ubuntu/dists/disco/pool/stable/amd64/docker-ce-cli_19.03.3~3-0~ubuntu-disco_amd64.deb"
wget "https://download.docker.com/linux/ubuntu/dists/disco/pool/stable/amd64/docker-ce_19.03.3~3-0~ubuntu-disco_amd64.deb"
sudo dpkg -i "containerd.io_1.2.6-3_amd64.deb"
sudo dpkg -i "docker-ce-cli_19.03.3~3-0~ubuntu-disco_amd64.deb"
sudo dpkg -i "docker-ce_19.03.3~3-0~ubuntu-disco_amd64.deb"

Then I issue the command:

 $ sudo apt-get -y install docker-ce
 Reading package lists... Done
 Building dependency tree       
 Reading state information... Done
 docker-ce is already the newest version (5:19.03.3~3-0~ubuntu-disco).
 0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

I believe Docker is installed now. I run:

 $ sudo docker run hello-world

Docker is correctly installed!

3.2 Configure Docker to run as regular user

I run these commands:

 $ sudo groupadd docker
 $ sudo gpasswd -a ${USER} docker
 $ sudo service docker restart

Now Docker should work without sudo command. To test, run:

 $ docker run hello-world

It works!


4. Download the build scripts

Issue the following commands:
 $ rm -rf ~/Downloads/openocd-xpack.git
 $ git clone --recurse-submodules https://github.com/xpack-dev-tools/openocd-xpack.git \
 ~/Downloads/openocd-xpack.git

The guide also mentions the --branch xpack-develop flag, but that one should not be used.


5. Adapt the build scripts

Now I stop following the guide at https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/README-BUILD.md , but I continue with the recommendations from @Tommy Murphy in the mails.

Navigate to the ~/Downloads/openocd-xpack.git/scripts folder:

 $ cd ~/Downloads/openocd-xpack.git/scripts

Now open the file defs-source.sh to edit it:

 $ gedit defs-source.sh

Add the following lines at the end of the file:

OPENOCD_GIT_URL=git://git.code.sf.net/p/openocd/code
OPENOCD_GIT_BRANCH=master
OPENOCD_GIT_COMMIT=HEAD

Some notes from @Tommy Murphy:

Just to clarify - by adding the following to .../scripts/defs-source.h at the end of the file you can build from the upstream//master repo:

OPENOCD_GIT_URL=git://git.code.sf.net/p/openocd/code
OPENOCD_GIT_BRANCH=master
OPENOCD_GIT_COMMIT=HEAD

@Liviu - yes, HEAD instead of a commit id works fine. :slightly_smiling_face:


6. Execute the build scripts

and then build as normal:

 $ bash ~/Downloads/openocd-xpack.git/scripts/build.sh --all 

(or instead of --all any combination of --linux32, --linux64, --win32, --win64 etc.).

The build outputs the binaries into ~/Work/openocd-0.10.0-14/deploy

Copy the binaries to a shared folder:

 $ cd ~/Work/openocd-0.10.0-14/deploy
 $ cp -R * /media/sf_ubuntu_shared
 $ cd /media/sf_ubuntu_shared
 $ ls
 xpack-openocd-0.10.0-14-linux-x32.tar.gz
 xpack-openocd-0.10.0-14-linux-x32.tar.gz.sha
 xpack-openocd-0.10.0-14-linux-x64.tar.gz
 xpack-openocd-0.10.0-14-linux-x64.tar.gz.sha
 xpack-openocd-0.10.0-14-win32-x32.zip
 xpack-openocd-0.10.0-14-win32-x32.zip.sha
 xpack-openocd-0.10.0-14-win32-x64.zip
 xpack-openocd-0.10.0-14-win32-x64.zip.sha