🗊Презентация Automatic testing. Training

OKTET Labs Test Environment HG Automatic Testing Training June 15, 2008 Konstantin Ushakov Konstantin.Ushakov@oktetlabs.ru

Plan Development Goals Architecture Structure Agents Engine Gateway testing

Development Goals QA of developed products Several pure-testing projects Socket API iSCSI Ethernet NIC firmware and drivers IPv6 ... Strong regression detection tool

Key features Black and white box testing Distributed testing Automatic testing Test coverage estimation Regression tracking tools Cross-platform testing Networking protocols support Strong scenarios parametrization system

Used in IPv4/IPv6 Socket API (more than 8000 tests); Ethernet switch testing; IEEE 802.2 (LLC), IEEE 802.1Q (VLAN); IEEE 802.1D (STP); ATM/AAL5 testing; Elements of 802.1X, WPA, WPA2, RADIUS client on Access Point; iSCSI testing; CLI testing; Testing SNMP MIBs; DHCP Client testing; CableHome1.0 some portal testing; Hyperlan2 MAC layer;

Base components Test Engine Run on Linux Build all components of TE Control of Test Agents Keep track of configuration changes Run tests Centralized log gathering

Test Agent Target-specific application Linux FreeBSD NetBSD Windows OpenSolaris Run on an auxiliary station or DUT Configuring station (interfaces,routes,etc.)‏

Test Agent Generates/Handles network traffic: Packets representation in ASN.1 format Packet iteration, inter packet interval, number of packets Sophisticated incoming traffic matching Supported packet formats: IEEE-802.3, IEEE-802.1D, IEEE-802.1Q, IEEE-802.2 ATM, AAL5 ARP, IPv4, IPv6, PPP, ICMPv4, UDP, TCP, DHCP, SNMP ...

Test Agent Sun RPC functionality socket API remote command line Curl File operations ... Daemons configuration: DHCP, DNS, FTP, TFTP, SNTP, LDAP, Radius, VTUN, 802.1X supplicant

Auxiliary Agent Black Box testing: Network protocols Interoperability External Interfaces (SNMP, CLI)‏ Access to external equipment: Sophisticated traffic generators Media-specific condition emulators

Embedded Agent White Box testing: Module testing API testing DUT-specific management

Proxy Agent Configure DUT Access to external equipment

Test Engine Components Dispatcher Runs/terminates all modules Handles user interrupt events Builder Based on autoconf/automake/m4 Cross compilation of Test Agents Logger RCF (Remote Control Facility)‏ Configurator Tester

Logger Provide unified logging interface for all TE components: Test Engine and Test Agents Time synchronization between Agents and Engine Sort out logs by timestamps Define a number of log levels Gather log dumps from Test Agents Extended format qualifiers (to log files, memory dumps, packets)‏ Save log messages into a binary file

Remote Control Facility Starts Test Agents according to configuration file or on-fly requests Provides access to services exported by Test Agents Controls concurrent access to Test Agents

Configurator Loads start-up configuration of Agents Keeps track of configuration changes Rolls back configuration to a specified point Restores configuration after Test Agent crash Save configurations in XML format

Configuration Tree /agent:Agt_A/interface:eth0 = /agent:Agt_A/interface:eth0/index: = 2 /agent:Agt_A/interface:eth0/link_addr: = 00:16:d3:26:98:91 /agent:Agt_A/interface:eth0/mtu: = 1500 /agent:Agt_A/interface:eth0/net_addr:192.168.56.1 = 24 /agent:Agt_A/interface:eth0/net_addr:192.168.56.1/broadcast: = 192.168.56.255 /agent:Agt_A/interface:eth0/net_addr:fe80::216:d3ff:fe26:9891 = 64 /agent:Agt_A/interface:eth0/net_addr:fec0::1 = 48 /agent:Agt_A/interface:eth0/status: = 1 … /agent:Agt_A/route:192.168.56.0|24 = 0.0.0.0 /agent:Agt_A/route:192.168.56.0|24/dev: = eth0 /agent:Agt_A/route:192.168.56.0|24/irtt: = 0 /agent:Agt_A/route:192.168.56.0|24/mtu: = 0 /agent:Agt_A/route:192.168.56.0|24/win: = 0 …

Tester Structured test suite organization Test iteration by parameters Test Agents can be automatically preconfigured before test run Requirement validation

TE auxiliary tools Test Coverage Estimation Report Generator Test Result Comparator

HG Testing Proxy agents Parametrization Networking capabilities Serial console logging Management interfaces VCM WebUI cutils

Configuration 1

Configuration 2

Questions?