2019-12-04

Okay, so this isn't promising:

~/src/xv6-intel
(0) <freeside:kyle> $ make qemu-nox
qemu-system-i386 -nographic -drive file=fs.img,index=1,media=disk,format=raw
-drive file=xv6.img,index=0,media=disk,format=raw -smp 2 -m 8M
xv6...
cpu1: starting 1
lapicid 0: panic: userinit: out of memory?
 801038fd 80102fba 0 0 0 0 0 0 0 0

That last -m 8M sets the memory size; the intel version of the code can't fit in 8MB of memory.

In fact, the lower bound on memory is 224MB. I don't yet know what drives this number, but I plan to investigate further.

The next step is to look into the whether the RISC-V version suffers the same fate. So, I followed the directions to get qemu-riscv built, pulled down the current version of xv6, and fired away. Right away:

~/src/xv6-riscv
(0) <freeside:kyle> $ make qemu MEM=8
qemu-system-riscv64 -machine virt -bios none -kernel kernel/kernel -m 8
-smp 3 -nographic -drive file=fs.img,if=none,format=raw,id=x0 -device
virtio-blk-device,drive=x0,bus=virtio-mmio-bus.0

xv6 kernel is booting

panic: kvmmap
QEMU: Terminated

No dice, but not surprising. It turns out 128M is the minimum. I guess I need to figure out where the memory is going in the kernel.

Also I found out about Zephyr today which looks interesting. Maybe something fun to mess around with the HiFive 1.

In order to dig into this, I need to figure out how this is getting built; what's going on during the compilation stage? The first step is

cd ${PROJ} build/lib && /opt/kendryte-toolchain/bin/riscv64-unknown-elf-g++
-DCONFIG_LOG_COLORS
-DCONFIG_LOG_ENABLE
-DCONFIG_LOG_LEVEL=LOG_VERBOSE
-DDEBUG=1
-DLOG_KERNEL
-DLV_CONF_INCLUDE_SIMPLE
-DNNCASE_TARGET=k210
-DTCB_SPAN_NO_CONTRACT_CHECKING
-DTCB_SPAN_NO_EXCEPTIONS
-D__riscv64
-I${PROJROOT}/lib/bsp/include
-I${PROJROOT}/lib/drivers/include
-I${PROJROOT}/lib/nncase/include
-I${PROJROOT}/lib/nncase/include/kernels/cpu
-I${PROJROOT}/lib/nncase/include/kernels/k210
-I${PROJROOT}/lib/nncase/include/kernels
-I${PROJROOT}/lib/nncase/include/kernels/neutral
-I${PROJROOT}/lib/nncase/include/runtime
-I${PROJROOT}/lib/nncase/include/runtime/cpu
-I${PROJROOT}/lib/nncase/include/runtime/k210
-I${PROJROOT}/lib/nncase/include/runtime/neutral
-I${PROJROOT}/lib/nncase/include/targets
-I${PROJROOT}/lib/utils/include
-I${PROJROOT}/third_party/xtl/include
-mcmodel=medany
-mabi=lp64f
-march=rv64imafc
-fno-common
-ffunction-sections
-fdata-sections
-fstrict-volatile-bitfields
-fno-zero-initialized-in-bss
-ffast-math
-fno-math-errno
-fsingle-precision-constant
-Os
-ggdb
-std=gnu++17
-Wall
-Werror=all
-Wno-error=unused-function
-Wno-error=unused-but-set-variable
-Wno-error=unused-variable
-Wno-error=deprecated-declarations
-Wno-multichar
-Wextra
-Werror=frame-larger-than=32768
-Wno-unused-parameter
-Wno-sign-compare
-Wno-error=missing-braces
-Wno-error=return-type
-Wno-error=pointer-sign
-Wno-missing-braces
-Wno-strict-aliasing
-Wno-implicit-fallthrough
-Wno-missing-field-initializers
-Wno-int-to-pointer-cast
-Wno-error=comment
-Wno-error=logical-not-parentheses
-Wno-error=duplicate-decl-specifier
-Wno-error=parentheses
-g
-o CMakeFiles/kendryte.dir/nncase/runtime/kernel_registry.cpp.obj
-c /home/kyle/src/kendryte-standalone-sdk/lib/nncase/runtime/kernel_registry.cpp

So I didn't really know what some of this meant so I started to look it up.

Starting with the mcmodel, I found The RISC-V Code Models post on the sifive site to be pretty helpful. Code models are, to quote from section 3.5.1 of the System V Application Binary Interface,

Code models define constraints for symbolic values that allow the compiler to generate better code. Basically code models differ in addressing (absolute versus position independent), code size, data size and address range. We define only a small number of code models that are of general interest:

And of course since this is the x86_64 ABI, "small number" is relative; there's seven code models defined.

So, what's a code model

To paraphrase, code models are a contract between the compiler and the linker - the compiler doesn't know where the code is going to end up, and we need to be able to tell the compiler which addressing modes are safe to use. This, in turn, lets it generate appropriate code. In Understanding the x64 code models, Eli Bendersky writes (referencing amd64):

A code model is a formal agreement between the programmer and the compiler, in which the programmer states his intentions for the size of the eventual program(s) the object file that's being currently compiled will get into [2].

Code models exist for the programmer to be able to tell the compiler: don't worry, this object will only get into non-huge programs, so you can use the fast RIP-relative addressing modes. Conversely, he can tell the compiler: this object is expected to be linked into huge programs, so please use the slow but safe absolute addressing modes with full 64-bit offsets.

What are the RISC-V code models?

The RISC-V ABI defines two code models:

  • medlow or medium-low requires that any global symbols be addressable by an signed 32-bit absolute address; technically, it means that address are generated using the lui address. All global symbols must have an address between 0 and 2GB.
  • medany or medium-any uses the auipc [1] instruction, which is (PC-) relative addressing scheme; therefore, global symbols must fit into any unsigned 32-bit address space.

-march

Though it comes later, we need to talk about this before the ABI part. It defines the architecture, and therefore the ISA. The Kendryte K210 uses the rv64imafc. I haven't found a good coverage of these yet, but I'll be looking for more information on it.

-mabi

The ABI flag

Specify integer and floating-point calling convention. ABI-string contains two parts: the size of integer types and the registers used for floating-point types.

In this case, we're compiling with the lp64f convention: l(ong) and p(ointers) are 64 bits, while floats are 32-bit (because the F registers are 32-bits). This is ISA-dependent, so it has to be read in the context of the -march argument. The current RISC-V GCC docs state that

The valid calling conventions are: ‘ilp32’, ‘ilp32f’, ‘ilp32d’, ‘lp64’, ‘lp64f’, and ‘lp64d’.

There's also a note that some combinations are illegal; for example,

'-march=rv32if -mabi=ilp32d' is invalid because the ABI requires 64-bit values be passed in F registers, but F registers are only 32 bits wide.

Like I mentioned, I still need to find out more about the various RISC-V architectures; I've ordered The RISC-V Reader: An Open Architecture Atlas that looks pretty good.

Hopefully tomorrow, I'll inspect the rest of the lines.

Some additional reading

Code models

[1]add upper immediate to pc: "UIPC forms a 32-bit offset from the 20-bit U-immediate, filling in the lowest 12 bits with zeros, adds this offset to the pc, then places the result in register rd."

Tags: ,