Test Plan for Linaro OpenEmbedded RPB release for Qualcomm™ Dragonboard™ 845c

Linaro Logo

Date: August 16, 2021

Executive Summary

This document describes the testing Linaro will carry out when making a release of the Linaro OpenEmbedded RPB release for Qualcomm™ Dragonboard™ 845c.

Hardware Platforms

Linaro provide releases for the following platforms from Qualcomm:

Board
Dragonboard845c

Tests definitions

Linaro uses LAVA test definition syntaxt to create and manage automated and manual tests. To learn more about LAVA test definition, you can refer to LAVA Documentation, in particular: Writing a Lava-Test Test Definition 1.0ΒΆ.

Linaro test cases are archived in the upstream Linaro Test Definitions repository here: https://github.com/Linaro/test-definitions/.

Automated Tests

Linaro runs automated testing in the LAVA lab.

This section describes the automated tests used from the Linaro Test Definitions repository.

1. linux-smoke

Run basic smoke tests.

Test definition: automated/linux/smoke/smoke.yaml
Test scope: functional

Parameter Value
SKIP_INSTALL False
TESTS pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat, lsblk

2. linux-usb-smoke

Basic USB test for Linaro Ubuntu images. The test examines all available USB devices and prints supported USB protocols and speed.

Test definition: automated/linux/usb-smoke/usb-smoke-test.yaml
Test scope: functional

Parameter Value
SKIP_INSTALL false

3. linux-toolchain-smoke

Simple test that checks whether gcc compilation works

Test definition: automated/linux/toolchain-smoke/toolchain-smoke.yaml
Test scope: functional

Parameter Value
STATIC False
SKIP_INSTALL False

4. linux-device-tree

Device tree test to check the folder structure. The test verifies that device-tree is available and contains model name of the board.

Test definition: automated/linux/device-tree/device-tree.yaml
Test scope: functional

5. meminfo

Collect the information in /proc/meminfo after boot.

Test definition: automated/linux/meminfo/meminfo.yaml
Test scope: functional

6. linux-dd-wr-speed

dd write/read speed test.

Test definition: automated/linux/dd-wr-speed/dd-wr-speed.yaml
Test scope: performance

Parameter Value
PARTITION
FS_TYPE
ITERATION 5
SKIP_INSTALL false

7. linux-sysbench

SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load. Current features allow to test fileio, cpu, memory, threads, mutex and oltp.

Test definition: automated/linux/sysbench/sysbench.yaml
Test scope: performance

Parameter Value
TESTS percpu cpu memory threads mutex fileio
NUM_THREADS NPROC
SKIP_INSTALL false

8. linux-device-read-perf

Use ‘hdparm -t’ to perform timings of block device reads for benchmark and comparison purposes.

Test definition: automated/linux/device-read-perf/device-read-perf.yaml
Test scope: performance

Parameter Value
DEVICE
SKIP_INSTALL False

9. linux-hackbench

Hackbench is both a benchmark and a stress test for the Linux kernel scheduler. It’s main job is to create a specified number of pairs of schedulable entities (either threads or traditional processes) which communicate via either sockets or pipes and time how long it takes for each pair to send data back and forth.

Test definition: automated/linux/hackbench/hackbench.yaml
Test scope: performance

Parameter Value
ITERATION 1000
TARGET host
DATASIZE 100
LOOPS 100
GRPS 10
FDS 20
PIPE false
THREADS false

10. linux-linpack

Linpack benchmark calculates fLoating point operations per second(FLOPS).

Test definition: automated/linux/linpack/linpack.yaml
Test scope: performance

Parameter Value
ARRAY_SIZE 200

11. linux-openssl-speed

Use openssl speed command to test the performance of cryptographic algorithms

Test definition: automated/linux/openssl/openssl-speed.yaml
Test scope: performance

Parameter Value
SKIP_INSTALL False

12. linux-stream-uniprocessor

STREAM benchmark is a simple synthetic benchmark program that measures sustainable memory bandwidth (in MB/s). This test case run STREAM on a single processor.

Test definition: automated/linux/stream/stream-uniprocessor.yaml
Test scope: performance

13. linux-pi-stress

pi_stress checks Priority Inheritence Mutexes and their ability to avoid Priority Inversion from occuring by running groups of threads that cause Priority Inversions.

Test definition: automated/linux/pi-stress/pi-stress.yaml
Test scope: functional

Parameter Value
DURATION 5m
MLOCKALL false
RR false
BACKGROUND_CMD

14. linux-pm-qa

Test power management (PM-QA). Currently, the test runs cpufreq, cpuidle, cpuhotplug, thermal and cputopology by default and individual test can be run by setting TESTS parameter in test job definition too. Test case documentation available in wiki: https://wiki.linaro.org/WorkingGroups/PowerManagement/Resources/TestSuite/PmQaSpecification

Test definition: automated/linux/pm-qa/pm-qa.yaml
Test scope: functional

Parameter Value
RELEASE pm-qa-v0.5.2
TESTS cpufreq cpuidle cpuhotplug thermal cputopology
SKIP_INSTALL false

15. linux-containers

Docker smoke test. When docker not pre-installed, docker-ce will be installed from upstream on supported distributions.

Test definition: automated/linux/docker/docker.yaml
Test scope: functional

Parameter Value
IMAGE aarch64/ubuntu

16. linux-wlan-smoke

Basic WLAN smoke tests

Test definition: automated/linux/wlan-smoke/wlan-smoke.yaml
Test scope: functional

Parameter Value
DEVICE wlan0
BOOT enabled

17. wlan-download

Basic WLAN download test, requires ip, iw, wpa_supplicant and dhclient

Test definition: automated/linux/wlan-download/wlan-download.yaml
Test scope: functional

Parameter Value
DEVICE wlan0
ETHERNET_DEVICE
SSID_NAME
SSID_PASSWORD
FILE_URL
FILE_CHECKSUM
TIME_DELAY 0s
NAMESERVER

18. linux-hci-smoke

Basic HCI smoke tests The test will run the following commands: – hciconfig up/down – hcitool scan – hcitool lescan (when le mode is enabled) – hcitool inq When testing LE mode some helper BT device must be in DUT’s proximity.

Test definition: automated/linux/hci-smoke/hci-smoke.yaml
Test scope: functional

Parameter Value
DEVICE hci0
BOOT enabled
MODE scan