Introduction to testing — Das U-Boot unknown version documentation
U-Boot has a large amount of code. This file describes how this code is
tested and what tests you should write when adding a new feature.
Running tests
To run most tests on sandbox, type this:
make check
in the U-Boot directory. Note that only the pytest suite is run using this
command.
Some tests take ages to run and are marked with @pytest.mark.slow. To run just
the quick ones, type this:
make qcheck
It is also possible to run just the tests for tools (patman, binman, etc.).
Such tests are included with those tools, i.e. no actual U-Boot unit tests are
run. Type this:
make tcheck
You can also run a selection tests in parallel with:
make pcheck
All of the above use the test/run script with a paremeter to select which tests
are run. See U-Boot pytest suite for more information.
Sandbox
U-Boot can be built as a user-space application (e. g. for Linux). This
allows test to be executed without needing target hardware. The ‘sandbox’
target provides this feature and it is widely used in tests.
See Sandbox tests for more information.
Pytest Suite
Many tests are available using the pytest suite, in test/py. This can run
either on sandbox or on real hardware. It relies on the U-Boot console to
inject test commands and check the result. It is slower to run than C code,
but provides the ability to unify lots of tests and summarise their results.
You can run the tests on sandbox with:
./test/py/test.py --bd sandbox --build
This will produce HTML output in build-sandbox/test-log.html
Some tests run with other versions of sandbox. For example sandbox_flattree
runs the tests with livetree (the hierachical devicetree) disabled. You can
also select particular tests with -k:
./test/py/test.py --bd sandbox_flattree --build -k hello
There are some special tests that run in SPL.
For this you need the sandbox_spl
build:
./test/py/test.py --bd sandbox_spl --build -k test_spl
See test/py/README.md for more information about the pytest suite.
See Sandbox tests for how to run tests directly (not through pytest).
tbot
Tbot provides a way to execute tests on target hardware. It is intended for
trying out both U-Boot and Linux (and potentially other software) on a
number of boards automatically. It can be used to create a continuous test
environment. See http://www.tbot.tools for more information.
When to write tests
If you add code to U-Boot without a test you are taking a risk. Even if you
perform thorough manual testing at the time of submission, it may break when
future changes are made to U-Boot. It may even break when applied to mainline,
if other changes interact with it. A good mindset is that untested code
probably doesn’t work and should be deleted.
You can assume that the Pytest suite will be run before patches are accepted
to mainline, so this provides protection against future breakage.
On the other hand there is quite a bit of code that is not covered with tests,
or is covered sparingly. So here are some suggestions:
-
If you are adding a new uclass, add a sandbox driver and a test that uses it
-
If you are modifying code covered by an existing test, add a new test case
to cover your changes -
If the code you are modifying has not tests, consider writing one. Even a
very basic test is useful, and may be picked up and enhanced by others. It
is much easier to add onto a test — writing a new large test can seem
daunting to most contributors.
See doc:tests_writing for how to write tests.
Future work
Converting existing shell scripts into pytest tests.
Using U-boot as production test strategy — really?
We hear periodically of customers using U-Boot as part of their test strategy. Most use U-boot as a Linux OS loader, and once the OS is loaded, they start a test application.
Others use the built-in test functions within U-Boot or build an application that is U-Boot compatible for loading/execution. I’d like to peel back this onion, so to speak, on these methods. I always need a practical application to think through the idea, to dissect or expose problems or pitfalls. So, for the sake of discussion, I’ll start on the ZedBoard using the Zynq-7020 SoC from Xilinx
The first area to understand is what is the state of the hardware? I’ll assume for the sake of argument that a good boundary-scan test suite has been applied to the hardware and we can move ahead expecting the SoC to be correctly fitted to the PCB and presumable operational.
Since we are examining a functional test strategy, we need to understand the boot process or when is the U-Boot accessible to operate? If you are not familiar with the Zynq-7000 SoC there is a boot sequence pictured below. This figure starts at the BootROM and completes at the Linux Kernel.
The BootROM is located within the SoC and not configurable by the user.
The BootROM determines whether the boot is secure or non-secure, performs some initialization of the system and clean-up. Next the ROM code determines (usually via switch settings) the desired boot mode (NAND, QSPI, JTAG) and loads the First Stage Boot Loader (FSBL) from the selected peripheral/interface into the On-Chip Memory (OCM) and jumps to the starting location of the FSBL.
The FSBL code does the hardware initialization, including the external memory, multiplex IO (MIO) and system clocks. This initialization is necessary to move to stage 2 where U-Boot would be loaded. So, it would appear the first point where the hardware is running from a functional aspect is after the FSBL at the end of Stage 1. However, that is not entirely true. The SoC can operate using the OCM and the JTAG interface only prior to hardware initialization. This does require a JTAG controller and careful management of the 292KB of OCM available.
Note: The use of a JTAG controller is highly recommended for U-Boot development and debug so there shouldn’t be an issue when considering a JTAG controller for test.
Then progressing to Stage 2, U-Boot is finally loaded. I not going to address what is needed to build U-Boot other than to say you will need a cross compiler to match the core (Arm) of the Zynq-7000 architecture and other tools. Getting the environment setup to build U-Boot is not trivial and is time-consuming. The time involved is inversely proportional to your experience with building Open System software. Once U-Boot is in a downloadable format, it can be placed in Non-Volatile storage, like an SD card, for downloading into DRAM.
Here is where I see a dilemma with using U-Boot for test. First I have to build U-Boot, and second have enough of the system operational, DDR (DDR controller/PHY) and SD flash interface etc. before testing can begin. Testing often involves non-operational system or minimally functionally operational hardware. The use of U-Boot expects a significant portion of the target hardware operational.
Second, assuming that you were successful in building U-Boot you now need to load it on the flash of the UUT.
Be sure to follow the below warning. If U-Boot is already installed and running on your board, you can use these instructions to download another U-Boot image to replace the current one.
Warning: Before you can install the new image, you must erase the current one. If anything goes wrong your board will be dead. It is strongly recommended that you have a backup of the old, working U-Boot image or you know how to install an image on a virgin system.
Next U-Boot has limited testing capabilities. It has memory commands, flash memory commands, execution control commands, download commands, and miscellaneous commands. Which sounds robust. However, take mtest which is the memory test function within U-Boot. It is a simple RAM test. The command structure is as follows:
mtest [start [end [pattern [iterations]]]]
So with scripting, which is also supported, one could create a sequence of tests with various patterns to test the address and data paths.
However, a warning that is within the U-Boot developer guide is that you need to be sure not to test operational space of U-Boot such as exception vectors, heap or stack space or U-Boot’s internal program code. Also, don’t use this in ROM or flash memory regions. You might crash U-Boot. So, the memory test presupposes that I have already tested the memory region where I expect to load and execute U-Boot. I can test memory with JTAG for the U-Boot region/footprint. But if the footprint is large it will impact the production beat rate.
Then there is a question in my mind about the diagnostic error messaging for mtest. It is not documented, but I assume that this is a pass/fail test. If so, the test developer would need to determine the cause of the failure, which might require other test function development, to provide diagnostic messages appropriate for the end consumer.
As I mentioned you could develop a sequence of tests to test memory for address and data (which have already been developed by countless engineering teams) but how would you create situational stress on the memory subsystem? Say induced noise?
So, in summary, the way I see it, some of the drawbacks of a U-boot based strategy are:
- It presumes that the SoC is assembled correctly to the board and operational
- DDR, SD flash, etc.
to load U-boot are operational - It presumes that U-boot has already been developed for the target
- U-boot utilities have their own set of constraints
- Impact on beat rate is high
- Diagnostic messaging and granularity is questionable
Imagine if there were a way to achieve a test coverage better than U-boot, but not need U-boot, and run before the FSBL. This way you could ensure the FSBL hardware initialization worked and if it didn’t it could indicate a problem within the FSBL code. It is not necessary to imagine these capabilities. ScanWorks® PFx tools use the SoC hardware resources like the processor and on chip memory (OCM) to provide memory test access at the bare metal. Great for prototype or production testing.
There are other topics that I hope to address in future blogs on U-Boot as a test strategy.
How to check a bootable flash drive, is it working, what type of bootloader does it have (regular or UEFI)?
Good day everyone!
From time to time, I sometimes get questions about checking a bootable flash drive: is it working, can it be used? BIOS / UEFI (details on this) to boot from a flash drive, install the recorded version of Windows, and work a little in it.
The fact is that in some cases errors can occur both during the installation of the system and after that. Agree, you are unlikely to be able to use the OS if it periodically crashes with blue screens… Windows OS…
👉 Help!
How to create a bootable Windows 10 flash drive under UEFI or BIOS — https://ocomp.info/sozdat-zagruz-fleshku-v-rufus.html
*
Method 1
Perhaps the most reliable way (as I said above) is to connect the USB flash drive to the USB port, restart the computer (laptop) and open the Boot Menu (boot menu). If the flash drive is bootable, you will see it in the menu list, and you will be able to select it and start downloading.
Boot Menu — choose a flash drive to boot
Simple, fast, reliable?! 👌
Method 2
One of the most popular programs for preparing and creating installation (bootable) flash drives WinSetupFromUSB also allows you to test and check their performance (does Windows boot from them?).
To do this:
- connect the flash drive to the USB port;
- run WinSetupFromUSB and select the connected drive from the list;
- check the box that you need «Test in QEMU» ;
- press button «GO» .
WinSetupFromUSB — checking the flash drive
After that, the program will automatically recreate the «real» working conditions of the flash drive and try to boot the OS from it. If everything is in order with the drive, you should see the Windows logo appear (as in my screenshot below).
Boot started — flash drive working (WinSetupFromUSB)
Method 3
MobaliveCD is a free program for checking bootable ISO images and flash drives. Allows literally in 2-3 mouse clicks to see how the installation flash drive will behave: whether the boot menu appears, whether the OS starts, etc.
I note that MobaliveCD does not need to be installed, but you need to run it as an administrator (just right-click on the downloaded file.
.. See the screenshot below).
Run as administrator
The main menu of the program is extremely simple:
- if you need to check the ISO image with Windows — click on the button «Run the LiveCD» ;
- if the flash drive is the button «Run the LiveUSB» (example below).
MobaLiveCD — flash drive test
Next, as a rule, you need to specify either a letter with a flash drive (or the location of the desired ISO file). I note that the program may ask you to specify where it will save its image (used for the test) — specify any of your disks (later this temporary file can be deleted).
Selecting the letter of the flash drive
Actually, if everything is in order with your bootable flash drive, the Windows logo will appear (installation has begun …).
Download started
Method 4
Special programs that virtually recreate a computer in your Windows (called virtual machines).
In such a virtual machine, you can install, for example, your old version of Windows 2000 and work in it without leaving your current Windows 10. Looking at the screenshot below, I think it will become more clear… 👇
A window is open in Windows 10 virtual machine with Windows XP (example of what it is about)
Of course, the virtual machine helps completely check the bootable USB flash drive: i.e. you will be able to see not only the installation process from it, but also the operation of the OS later (there will be / will not be errors)! In my opinion, this is one of the most reliable options!
In general, the topic of setting up a virtual machine is quite large, so if you are interested, see the article, the link to which is given below.
👉 Help!
How to run old Windows on a new operating system: setting up a VirtualBox virtual machine — https://ocomp.info/kak-zap-staruyu-windows-virtualbox.
html
written to work from under UEFI or BIOS?
First, , pay attention to the parameters of the program in which you are creating a bootable USB flash drive. For example, in Rufus, when creating a flash drive, you manually specify the target system: BIOS or UEFI…
RUFUS — creating a flash drive
Secondly , try entering the boot menu (when you turn on the laptop PC) and see if your system sees the flash drive (if you have UEFI, «UEFI:» will be lit opposite your flash drive , see screenshot below). This is a quick and reliable way to find out the type of flash drive…
Boot menu example / ASUS laptop
Third , try to open Disk Management (this is a built-in utility in Windows). Most often, the FAT32 file system is used in flash drives under UEFI.
File system
You can also open the properties of the drive (note that you need to right-click the drive menu).
Disk properties
In the tab «Volumes» there can be 2 types of partition:
- Master boot record (MBR) is a flash drive under the BIOS;
- The partition GUID table is a UEFI flash drive.
MBR (left) — GUID (right)
*
That’s all for now…
Comments are welcome.
👋
RSS (how to read Rss)
Useful software:
- Video Montage
Great software for making your first videos (all step by step!).
Even a beginner can make a video!
- computer accelerator
Program for cleaning Windows from «garbage» (deletes temporary files, speeds up the system, optimizes the registry).
Other entries:
Roblox game test: Are you a noob or a pro?
in Video Game Tests, Roblox
Game Tests
published
Schoolboy
Answer the questions and find out how well you understand the Roblox game.
Roblox is an open-world online game where you can build a variety of buildings, explore alien worlds and chat with like-minded people. Think you’re an expert on it? Then let’s test your Roblox knowledge!
Good luck!
-
-
David Baszuki
-
John Doe
-
David Bowie
-
Carl Sagan
-
-
-
Quickman
-
Sandman
-
Builderman
-
Slenderman
-
-
-
Rodollars
-
Rocoins
-
-
Xbox One
-
Android
-
-
-
Builder
-
Builderman
-
Builderbro
-
-
-
Dyna Blocks
-
Noblox
-
Soblox
-
-
-
Coolcity
-
Minecity
-
Meepcity
-
Terrorcity
-
-
-
Roblox Core
-
Roblox Neo
-
Roblox Nexus
-
Roblox Studio
-
Did you like it?
3.
7k Points
Yes No
robloxgamerobloxtest
Don’t miss
-
in Minecraft
Tests
Minecraft 9 Difficult Quiz0435
This is not a simple, but a difficult test for the game Minecraft (and not only). There will be 10 questions, each of them is difficult. Take the Quiz Question from […] More
-
in Brawl Stars General Knowledge Quiz | Brawl Stars, Tests for the game Brawl Stars | Brawl Stars
How well do you know Brawl Stars? Test for PRO players
Do you like to play Brawl Stars? Are you the coolest among your company? Can you answer all the questions in this quiz? Test yourself! Good luck! Take the test […] More
-
in Tests for Vikings: Clan Wars
Quiz: How well do you know Vikings: Clan Wars?
Played a Viking toy? Then try to answer all the questions of the test and evaluate your knowledge in this game. Take the test Question from Which screenshots […] More
-
in Brawl Stars Tests | Brawl Stars
If you answer all the questions correctly, then you are a Brawl Stars expert!
Do you know a game like Brawl Stars? So let’s check how much!) Good luck! Take the quiz Question from When was Bravl’s NOT global release? […] More
-
in Who Are You From, Video Game Tests, Dota 2 Tests | Dota 2
Quiz: Which Dota 2 Hero Are You?
Take the test and find out who you would be in the world of Dota 2.
Take the test Question from Which attribute do you prefer? Strength Agility Intelligence Question from […] More -
in Video Game Tests, Dota 2 Tests | Dota 2
Test: Name the hero of Dota
In this test, you will need to guess the heroes of the popular game Dota by a piece of costume or weapon. The test is suitable only for the most real gamers who really […] More
-
in Brawler Tests, Brawl Stars Tests | Brawl Stars
Test: Only a genius can name all 10 brawlers!
We invite you to take this test and test your abilities. Answer all questions about Brawl Stars and see how well you know Brawl Stars! […] More
-
in Minecraft
Tests
Minecraft hostile mobs easy test
Let’s check how well do you know Minecraft mobs? Guess the hostile mob from the description. Good luck! Take the quiz Question from Huge white flying hostile mob inhabiting […] More
-
in Who are you from, Tests for the game Terraria
Quiz: Which Terraria Boss Are You?
Terraria game is quite an old game like Minecraft.
