Skip to content

Commit f430d60

Browse files
committed
crash: add kernel testing against vmcores
One of the things that has had a serious negative effect on the quality of crash-python is the inability to do automated testing on a broad variety of kernels and vmcores. Often, we have to run our tests by hand. More often, we get reports from new users that something broke unexpectedly. This commit adds real unit testing against real kernels and vmcores. It includes a few test cases that will need to be extended further. The policy moving forward will be that new features will require a matching test case that passes across a variety of kernels. Signed-off-by: Jeff Mahoney <jeffm@suse.com>
1 parent 34439c4 commit f430d60

20 files changed

Lines changed: 1377 additions & 2 deletions

‎INTERNALS.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
2+
# GDB
3+
4+
## Python contexts within GDB
5+
6+
Each time gdb enters the Python interpreter it establishes a context.
7+
Part of the context includes what architecture gdb believes it is
8+
debugging ('gdbarch') and that is passed into the context. If anything
9+
changes the gdbarch in that Python context, it won't be visible to any
10+
subsequent Python code until a new session is established.
11+
12+
When gdb starts up on x86_64, it uses a gdbarch of i386 -- with 32-bit words
13+
and pointers. Only when we load an executable or target does it switch
14+
to i386:x86_64.
15+
16+
The effect of this is that any code that relys on type information *must*
17+
be executed in a separate context from the one that loaded the executable
18+
and/or taret. Otherwise, any built-in types that are pointers or `long`
19+
based will use the 32-bit sizes.

‎TESTING.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Testing
2+
3+
## Summary
4+
5+
There are unit tests in the tests/ dir that are standalone and useful for
6+
testing basic functionality.
7+
8+
There are unit tests in the kernel-tests dir that require configuration,
9+
kernel images, debuginfo, and vmcores to use.
10+
11+
## Configuration
12+
13+
The configuration for each kernel/vmcore to be tested goes in a .ini file
14+
with the following format. All fields except kernel and vmcore are
15+
optional, and defaults will be used. A kernel missing debuginfo cannot
16+
be used for testing. Missing modules will mean module-specific tests
17+
will be skipped.
18+
19+
```[test]
20+
kernel=/path/to/kernel
21+
vmcore=/path/to/vmcore
22+
vmlinux_debuginfo=/path/to/vmlinux-debuginfo
23+
modules=/path/to/modules
24+
module_debuginfo_path=/path/to/module/debuginfo
25+
root=/root/for/tree/searches```
26+
27+
The optional fields match those defined in crash.kernel.CrashKernel.
28+
29+
Example 1:
30+
```[test]
31+
kernel=/var/crash/2019-04-23-11:35/vmlinux-4.12.14-150.14-default.gz
32+
vmcore=/var/crash/2019-04-23-11:35/vmcore```
33+
34+
In this example, the kernel and debuginfo packages are installed in the
35+
default locations and will be searched automatically.
36+
37+
Example 2:
38+
```[test]
39+
kernel=/var/crash/2019-04-23-11:35/vmlinux-4.12.14-150.14-default.gz
40+
vmcore=/var/crash/2019-04-23-11:35/vmcore
41+
root=/var/cache/crash-setup/leap15/4.12.14-150.14-default
42+
```
43+
44+
In this example, the kernel and debuginfo packages are installed under
45+
/var/cache/crash-setup/leap15/4.12.14-150.14-default and so we only
46+
specify a root directory.
47+
48+
## Running
49+
50+
The script `test-all.sh` when run with no options will execute only
51+
the standalone tests. The script takes a list of the .ini files
52+
described above and will execute the kernel tests against those
53+
configurations immediately after the standalone tests.
54+
55+
Example:
56+
```sh test-all.sh kernel-test-configs/4.12.14-150.14-default.ini kernel-test-configs/5.1.0-rc7-vanilla.ini```
57+
or
58+
```sh test-all.sh kernel-test-configs/*.ini```
59+
60+
Each configuration will execute independently from one another.
61+

‎crash/commands/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ def __init__(self, name, parser=None):
3737
self.commands[self.name] = self
3838
gdb.Command.__init__(self, self.name, gdb.COMMAND_USER)
3939

40-
def invoke_uncaught(self, argstr, from_tty):
40+
def invoke_uncaught(self, argstr, from_tty=False):
4141
argv = gdb.string_to_argv(argstr)
4242
args = self.parser.parse_args(argv)
4343
self.execute(args)

‎kernel-tests/decorators.py‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# -*- coding: utf-8 -*-
2+
# vim:set shiftwidth=4 softtabstop=4 expandtab textwidth=79:
3+
import unittest
4+
import gdb
5+
6+
def skip_with_type(typename):
7+
try:
8+
gdbtype = gdb.lookup_type(typename)
9+
return unittest.skip(f"found type {typename}")
10+
except gdb.error:
11+
pass
12+
13+
return lambda func: func
14+
15+
def skip_without_type(typename):
16+
try:
17+
gdbtype = gdb.lookup_type(typename)
18+
except gdb.error:
19+
return unittest.skip(f"missing type {typename}")
20+
21+
return lambda func: func
22+
23+
def skip_with_symbol(symname):
24+
symbol = gdb.lookup_symbol(symname, None)[0]
25+
if symbol is not None:
26+
return unittest.skip(f"found symbol {symname}")
27+
28+
return lambda func: func
29+
30+
def skip_without_symbol(symname):
31+
symbol = gdb.lookup_symbol(symname, None)[0]
32+
if symbol is None:
33+
return unittest.skip(f"missing symbol {symname}")
34+
35+
return lambda func: func
36+
37+
def has_super_blocks(name):
38+
from crash.subsystem.filesystem import for_each_super_block
39+
for sb in for_each_super_block():
40+
if sb['s_type']['name'].string() == name:
41+
return True
42+
return False
43+
44+
can_test = {}
45+
46+
def skip_with_supers(name):
47+
if not name in can_test:
48+
can_test[name] = has_super_blocks(name)
49+
50+
if not can_test[name]:
51+
return lambda func: func
52+
53+
return unittest.skip(f"{name} file systems in image")
54+
55+
def skip_without_supers(name):
56+
if not name in can_test:
57+
can_test[name] = has_super_blocks(name)
58+
59+
if can_test[name]:
60+
return lambda func: func
61+
62+
return unittest.skip(f"no {name} file systems in image")
Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# -*- coding: utf-8 -*-
2+
# vim:set shiftwidth=4 softtabstop=4 expandtab textwidth=79:
3+
import unittest
4+
import gdb
5+
import io
6+
import sys
7+
8+
from decorators import skip_with_symbol, skip_without_symbol
9+
10+
from crash.commands.dmesg import LogCommand
11+
from crash.commands import CommandError, CommandLineError
12+
13+
class TestCommandsLog(unittest.TestCase):
14+
def setUp(self):
15+
self.stdout = sys.stdout
16+
sys.stdout = io.StringIO()
17+
self.command = LogCommand("dmesg")
18+
19+
def tearDown(self):
20+
sys.stdout = self.stdout
21+
22+
def output(self):
23+
return sys.stdout.getvalue()
24+
25+
def test_dmesg(self):
26+
"""`dmesg' produces valid output"""
27+
self.command.invoke_uncaught("")
28+
output = self.output()
29+
self.assertTrue(len(output.split("\n")) > 2)
30+
31+
def test_dmesg_bad_option(self):
32+
"""`dmesg -x` raises CommandLineError"""
33+
with self.assertRaises(CommandLineError):
34+
self.command.invoke_uncaught("-x")
35+
36+
def test_dmesg_t(self):
37+
"""`dmesg' produces valid output"""
38+
self.command.invoke_uncaught("-t")
39+
output = self.output()
40+
self.assertTrue(len(output.split("\n")) > 2)
41+
42+
@skip_without_symbol('log_first_seq')
43+
def test_dmesg_d(self):
44+
"""`dmesg -d' produces valid output"""
45+
self.command.invoke_uncaught("-d")
46+
output = self.output()
47+
self.assertTrue(len(output.split("\n")) > 2)
48+
49+
@skip_with_symbol('log_first_seq')
50+
def test_dmesg_d(self):
51+
"""`dmesg -d' raises CommandError with unstructured log"""
52+
with self.assertRaises(CommandError):
53+
self.command.invoke_uncaught("-d")
54+
55+
def test_dmesg_m(self):
56+
"""`dmesg -m ' produces valid output"""
57+
self.command.invoke_uncaught("-m")
58+
output = self.output()
59+
self.assertTrue(len(output.split("\n")) > 2)
60+
61+
def test_dmesg_tm(self):
62+
"""`dmesg -t -m' produces valid output"""
63+
self.command.invoke_uncaught("-t -m")
64+
output = self.output()
65+
self.assertTrue(len(output.split("\n")) > 2)
66+
67+
@skip_without_symbol('log_first_seq')
68+
def test_dmesg_td(self):
69+
"""`dmesg -t -d' produces valid output"""
70+
self.command.invoke_uncaught("-t -d")
71+
output = self.output()
72+
self.assertTrue(len(output.split("\n")) > 2)
73+
74+
@skip_without_symbol('log_first_seq')
75+
def test_dmesg_dm(self):
76+
"""`dmesg -m -d' produces valid output"""
77+
self.command.invoke_uncaught("-m -d")
78+
output = self.output()
79+
self.assertTrue(len(output.split("\n")) > 2)
80+
81+
@skip_without_symbol('log_first_seq')
82+
def test_dmesg_tdm(self):
83+
"""`dmesg -t -d -m' produces valid output"""
84+
self.command.invoke_uncaught("-t -d -m")
85+
output = self.output()
86+
self.assertTrue(len(output.split("\n")) > 2)
87+
88+
@skip_with_symbol('log_first_seq')
89+
def test_dmesg_td(self):
90+
"""`dmesg -t -d' raises CommandError with unstructured log"""
91+
with self.assertRaises(CommandError):
92+
self.command.invoke_uncaught("-t -d")
93+
94+
@skip_with_symbol('log_first_seq')
95+
def test_dmesg_dm(self):
96+
"""`dmesg -m -d' raises CommandError with unstructured log"""
97+
with self.assertRaises(CommandError):
98+
self.command.invoke_uncaught("-m -d")
99+
100+
@skip_with_symbol('log_first_seq')
101+
def test_dmesg_tdm(self):
102+
"""`dmesg -t -d -m' raises CommandError with unstructured log"""
103+
with self.assertRaises(CommandError):
104+
self.command.invoke_uncaught("-t -d -m")
105+

‎kernel-tests/test_commands_kmem.py‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# -*- coding: utf-8 -*-
2+
# vim:set shiftwidth=4 softtabstop=4 expandtab textwidth=79:
3+
import unittest
4+
import gdb
5+
import io
6+
import sys
7+
8+
from decorators import skip_without_symbol
9+
from decorators import skip_with_symbol
10+
11+
from crash.commands.kmem import KmemCommand
12+
from crash.commands import CommandLineError, CommandError
13+
14+
class TestCommandsKmem(unittest.TestCase):
15+
def setUp(self):
16+
self.stdout = sys.stdout
17+
sys.stdout = io.StringIO()
18+
self.command = KmemCommand("kmem")
19+
20+
def tearDown(self):
21+
sys.stdout = self.stdout
22+
23+
def output(self):
24+
return sys.stdout.getvalue()
25+
26+
def test_kmem_empty(self):
27+
with self.assertRaises(CommandLineError):
28+
self.command.invoke_uncaught("")
29+
30+
def test_kmem_invalid(self):
31+
"""`kmem' returns error"""
32+
with self.assertRaises(CommandLineError):
33+
self.command.invoke_uncaught("invalid")
34+
35+
@unittest.skip("takes a huge amount of time on a real core")
36+
def test_kmem_s(self):
37+
"""`kmem -s' produces valid output"""
38+
self.command.invoke_uncaught("-s")
39+
output = self.output()
40+
self.assertTrue(len(output.split("\n")) > 2)
41+
42+
def test_kmem_s_inode_cache(self):
43+
"""`kmem -s inode_cache' produces valid output"""
44+
self.command.invoke_uncaught("-s inode_cache")
45+
output = self.output()
46+
self.assertTrue(len(output.split("\n")) > 2)
47+
48+
def test_kmem_s_unknown_cache(self):
49+
"""`kmem -s unknown_cache' raises CommandError"""
50+
with self.assertRaises(CommandError):
51+
self.command.invoke_uncaught("-s unknown_cache")
52+
53+
def test_kmem_sz(self):
54+
"""`kmem -s -z' raises CommandLineError"""
55+
with self.assertRaises(CommandLineError):
56+
self.command.invoke_uncaught("-s -z")
57+
58+
def test_kmem_sz_valid_cache(self):
59+
"""`kmem -s -z' raises CommandLineError"""
60+
with self.assertRaises(CommandLineError):
61+
self.command.invoke_uncaught("-s inode_cache -z")
62+
63+
def test_kmem_sz_invalid_cache(self):
64+
"""`kmem -s unknown_cache -z' raises CommandLineError"""
65+
with self.assertRaises(CommandLineError):
66+
self.command.invoke_uncaught("-s unknown_cache -z")
67+
68+
def test_kmem_sv(self):
69+
"""`kmem -s -V' raises CommandLineError"""
70+
with self.assertRaises(CommandLineError):
71+
self.command.invoke_uncaught("-s -V")
72+
73+
def test_kmem_sv_valid_cache(self):
74+
"""`kmem -s inode_cache -V' raises CommandLineError"""
75+
with self.assertRaises(CommandLineError):
76+
self.command.invoke_uncaught("-s inode_cache -V")
77+
78+
def test_kmem_sv_invalid_cache(self):
79+
"""`kmem -s unknown_cache -V' raises CommandLineError"""
80+
with self.assertRaises(CommandLineError):
81+
self.command.invoke_uncaught("-s unknown_cache -V")
82+
83+
def test_kmem_z(self):
84+
"""`kmem -z' produces valid output"""
85+
self.command.invoke_uncaught("-z")
86+
output = self.output()
87+
self.assertTrue(len(output.split("\n")) > 2)
88+
89+
def test_kmem_z_invalid(self):
90+
"""`kmem -z invalid' raises CommandLineError"""
91+
with self.assertRaises(CommandLineError):
92+
self.command.invoke_uncaught("-z invalid")
93+
94+
@skip_without_symbol('vm_stat')
95+
def test_kmem_v(self):
96+
"""`kmem -V' produces valid output"""
97+
self.command.invoke_uncaught("-V")
98+
output = self.output()
99+
self.assertTrue(len(output.split("\n")) > 0)
100+
101+
@skip_with_symbol('vm_stat')
102+
def test_kmem_v_unimplemented(self):
103+
"""`kmem -V' raises CommandError due to missing symbol"""
104+
with self.assertRaises(CommandError):
105+
self.command.invoke_uncaught("-V")
106+
107+
def test_kmem_v_invalid(self):
108+
"""`kmem -V invalid' raises CommandLineError"""
109+
with self.assertRaises(CommandLineError):
110+
self.command.invoke_uncaught("-V invalid")
111+
112+
def test_kmem_vz(self):
113+
"""`kmem -V -z' raises CommandLineError"""
114+
with self.assertRaises(CommandLineError):
115+
self.command.invoke_uncaught("-V -z")
116+
117+
def test_kmem_svz(self):
118+
"""`kmem -V -z -s' raises CommandLineError"""
119+
with self.assertRaises(CommandLineError):
120+
self.command.invoke_uncaught("-V -z -s")
121+
122+
def test_kmem_svz_valid_cache(self):
123+
"""`kmem -V -z -s inode_cache' raises CommandLineError"""
124+
with self.assertRaises(CommandLineError):
125+
self.command.invoke_uncaught("-V -z -s inode_cache")
126+
127+
def test_kmem_svz_invalid_cache(self):
128+
"""`kmem -V -z -s unknown_cache' raises CommandLineError"""
129+
with self.assertRaises(CommandLineError):
130+
self.command.invoke_uncaught("-V -z -s unknown_cache")

0 commit comments

Comments
 (0)