Zeren Li

Zeren Li

George

Computer Engineering Student

USB Armory MkII Tutorial 4

10 minute read

Published:

Last update: 2026-09-07

How to Use USB Armory Mk II (Part 4: GoTEE + Bear Metal Execution + Cache management)

Build and install GoTEE-example

This setup uses:

  • TamaGo Go toolchain: tamago-go1.24.6
  • GoTEE-example commit: df60168
  • Target: USB armory

1. Install the TamaGo Go 1.24.6 toolchain

cd /mnt/linuxstorage/usbarmory

git clone \
    --branch tamago-go1.24.6 \
    --depth 1 \
    https://github.com/usbarmory/tamago-go.git \
    tamago-go-1.24.6

cd tamago-go-1.24.6/src
./all.bash

Set the TamaGo compiler path:

export TAMAGO=/mnt/linuxstorage/usbarmory/tamago-go-1.24.6/bin/go

Verify that the TamaGo ARM target is available:

$TAMAGO version
$TAMAGO tool dist list | grep tamago

The output should include:

tamago/arm

2. Check out the compatible GoTEE-example revision

cd /mnt/linuxstorage/usbarmory/GoTEE-example

git checkout df60168

Verify the revision:

git rev-parse --short HEAD

Expected:

df60168

3. Download the pinned Go dependencies

Do not run:

go get -u github.com/usbarmory/tamago@latest

because this would replace the TamaGo dependency expected by this GoTEE-example revision with a newer, potentially incompatible version.

Instead, use the dependency versions already pinned by go.mod and go.sum:

$TAMAGO mod download

If go.sum has previously been modified by a newer TamaGo version, restore the files from commit df60168 first:

git restore --source=HEAD -- go.mod go.sum

$TAMAGO mod download

4. Build GoTEE

Set the target:

export TARGET=usbarmory
export TAMAGO=/mnt/linuxstorage/usbarmory/tamago-go-1.24.6/bin/go
export GOTOOLCHAIN=local

Then build:

make clean

make nonsecure_os_go
make trusted_applet_go
make trusted_os

Using GOTOOLCHAIN=local prevents Go from automatically downloading and switching to a newer upstream Go compiler, which would not provide the TamaGo-specific GOOS=tamago target.

5. Build configuration

The intended version combination is:

TamaGo compiler:
    tamago-go1.24.6

GoTEE-example:
    df60168

Target:
    usbarmory

Keep the versions pinned when reproducing this build. In particular, avoid updating github.com/usbarmory/tamago to @latest, because newer TamaGo releases use a different runtime/toolchain interface.

6. Boot the GoTEE-example

Unplug and replug usbarmoy

cd ~/Desktop/GoTEE-example/bin
sudo $HOME/go/bin/armory-boot-usb -i trusted_os_usbarmory.imx

Bring up the USB network on your host

# find the new interface (often usb0 or an enx* name)
ip link


# result
# USB-Ethernet gadget is enx1a5589a26942 (that’s the typical “enx…” name Linux gives to USB NICs
enx1a5589a26942: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 1a:55:89:a2:69:42 brd ff:ff:ff:ff:ff:ff

# pick the USB NIC
IF=enx1a5589a26942

# clean slate + set your host IP
sudo ip addr flush dev $IF
sudo ip addr add 10.0.0.2/24 dev $IF
sudo ip link set $IF up

# quick check
ip addr show dev $IF
ping -c 3 10.0.0.1

If ping works, SSH in:

ssh gotee@10.0.0.1

ssh issue

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:MNlLQ8Py8VcK4h3RxDwak2BFxDO6asF9vFY2CY8fPdA.
Please contact your system administrator.
Add correct host key in /home/lizeren/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/lizeren/.ssh/known_hosts:8
  remove with:
  ssh-keygen -f "/home/lizeren/.ssh/known_hosts" -R "10.0.0.1"
Host key for 10.0.0.1 has changed and you have requested strict checking.
Host key verification failed.

GoTEE unikernel generates a new SSH host key each time you boot it, so it no longer matches the one stored from a previous session.

You can safely remove the old key and reconnect:

ssh-keygen -f "/home/lizeren/.ssh/known_hosts" -R "10.0.0.1"

Then try again:

ssh gotee@10.0.0.1

Freezing problem

Sometimes the GoTEE will freeze and not respond to SSH. This is because the USB Armory firmware used 10.0.0.1 as the IP address for the GoTEE while the WI-FI router also uses 10.0.0.1. Linux routed traffic for that address through Wi‑Fi instead of the USB interface.

    1. So we should changed the USB Armory firmware address from 10.0.0.1 to 10.77.0.1 in trusted_os_usbarmory/main.go.
    1. Rebuilt bin/trusted_os_usbarmory.imx
sudo nmcli connection add \
    type ethernet \
    ifname enx1a5589a26942 \
    con-name usb-armory \
    ipv4.method manual \
    ipv4.addresses 10.77.0.2/24 \
    ipv4.never-default yes \
    ipv6.method disabled

sudo nmcli connection up usb-armory

What the settings mean:

  • ifname enx1a5589a26942: binds the profile to the USB Armory network adapter.
  • ipv4.addresses 10.77.0.2/24: assigns the host side of the USB connection.
  • The USB Armory uses 10.77.0.1.
  • ipv4.never-default yes: prevents this connection from replacing your Wi‑Fi default route.
  • ipv6.method disabled: disables unused IPv6 on this private link.

Verify it:

  nmcli connection show usb-armory

Check the assigned address:

  ip -brief address show dev enx1a5589a26942

Expected:

  enx1a5589a26942  UNKNOWN  10.77.0.2/24

Check routing:

  ip route get 10.77.0.1

Expected:

  10.77.0.1 dev enx1a5589a26942 src 10.77.0.2

Then connect:

  ssh gotee@10.77.0.1

Go application: Cache + Performance Counter

add this code to nonsecure_os_go/ and named it cache_timer.go

//go:build tamago && arm

package main

import (
    "github.com/usbarmory/tamago/arm"
)

func CacheTimerDemo() {
    cpu := arm.CPU{}
    cpu.EnableSMP()
    cpu.EnableCache()
    cpu.InitGenericTimers(0, 0)

    start := cpu.Counter()
    for i := 0; i < 1_000_000; i++ {}
    end := cpu.Counter()

    cpu.FlushDataCache()
    cpu.FlushInstructionCache()
    println("Cycle counter delta:", end-start)
}

now rebuild the GoTEE

cd GoTEE-example


export TARGET=usbarmory
make nonsecure_os_go
make trusted_applet_go
make trusted_os
#don't forget to unplug and replug the USB Armory
sudo $HOME/go/bin/armory-boot-usb -i bin/trusted_os_usbarmory.imx

# pick the USB NIC
IF=enx1a5589a26942

# clean slate + set your host IP
sudo ip addr flush dev $IF
sudo ip addr add 10.0.0.2/24 dev $IF
sudo ip link set $IF up

ssh-keygen -f "/home/lizeren/.ssh/known_hosts" -R "10.0.0.1"
ssh gotee@10.0.0.1

Now inside the GoTEE command shell, run

gotee						 # TrustZone example w/ TamaGo unikernels

Yuo should see the performance counter output.

Cycle counter delta: 35560

Examine cache disable

//go:build tamago && arm

package gotee

import (
	"log"
	"github.com/usbarmory/tamago/arm"
)

func CacheTimerDemo() {
	cpu := arm.CPU{}
	cpu.EnableSMP()
	cpu.EnableCache()
	cpu.InitGenericTimers(0, 0)

	// Create a memory array to access
	const arraySize = 1000
	data := make([]uint32, arraySize)
	
	// Initialize the array
	for i := 0; i < arraySize; i++ {
		data[i] = uint32(i)
	}

	// ===== Measure WITH cache enabled =====
	log.Println("=== Testing WITH cache enabled ===")
	
	start := cpu.Counter()
	var sum uint32
	for i := 0; i < 10000; i++ {
		sum += data[i%arraySize]  // Memory access
	}
	end := cpu.Counter()
	
	cyclesWithCache := end - start
	log.Println("Cycles WITH cache:", cyclesWithCache)
	log.Println("Sum (to prevent optimization):", sum)

	// ===== Disable cache and flush =====
	log.Println("\n=== Disabling cache ===")
	cpu.FlushDataCache()
	cpu.FlushInstructionCache()
	cpu.DisableCache()
	log.Println("Cache disabled")

	// ===== Measure WITHOUT cache =====
	log.Println("\n=== Testing WITHOUT cache ===")
	
	start = cpu.Counter()
	sum = 0
	for i := 0; i < 10000; i++ {
		sum += data[i%arraySize]  // Same memory access pattern
	}
	end = cpu.Counter()
	
	cyclesWithoutCache := end - start
	log.Println("Cycles WITHOUT cache:", cyclesWithoutCache)
	log.Println("Sum (to prevent optimization):", sum)

	// ===== Compare results =====
	log.Println("\n=== Comparison ===")
	log.Println("Cycles WITH cache:   ", cyclesWithCache)
	log.Println("Cycles WITHOUT cache:", cyclesWithoutCache)
	if cyclesWithoutCache > cyclesWithCache {
		slowdown := float64(cyclesWithoutCache) / float64(cyclesWithCache)
		log.Printf("Cache disabled is %.2fx slower\n", slowdown)
	}

	// Get CPU mode
	mode := cpu.Mode()
	log.Println("CPU current mode is:", mode)
}

Flush + Reload

https://github.com/lizeren/usbarmory-gotee/blob/main/trusted_os_usbarmory/internal/cache_timer.go

You May Also Enjoy

Performance counter 4

25 minute read

Published:

In this post we will learn what is max sampling rate parameters, sample overflow interrupts between kernel ticks and how linux kernel dynamically adjusts the max sample rate when it detects that an interrupt from sample overflow is taking too long.

The Pointer Mistake

9 minute read

Published:

pointer pointing to a value vs pointer pointing to a pointer

If a function is ever called

12 minute read

Published:

Given a large C project and I want to know if a vulnerable function is ever being used by the test cases, how to find it?

GCC FunctionInstrumentation

16 minute read

Published:

In this post, we explore how the -finstrument-functions flag and a custom GCC plugin can be used to intercept function entry and exit at compile time. We then examine a less obvious case: what happens when the function we want to instrument is not defined in the main binary, but instead lives in a shared library.