]> bbs.cooldavid.org Git - net-next-2.6.git/blame - Documentation/fault-injection/fault-injection.txt
stmmac: tidy-up stmmac_priv structure
[net-next-2.6.git] / Documentation / fault-injection / fault-injection.txt
CommitLineData
de1ba09b
AM
1Fault injection capabilities infrastructure
2===========================================
3
4See also drivers/md/faulty.c and "every_nth" module option for scsi_debug.
5
6
7Available fault injection capabilities
8--------------------------------------
9
10o failslab
11
12 injects slab allocation failures. (kmalloc(), kmem_cache_alloc(), ...)
13
14o fail_page_alloc
15
16 injects page allocation failures. (alloc_pages(), get_free_pages(), ...)
17
18o fail_make_request
19
5d0ffa2b 20 injects disk IO errors on devices permitted by setting
de1ba09b
AM
21 /sys/block/<device>/make-it-fail or
22 /sys/block/<device>/<partition>/make-it-fail. (generic_make_request())
23
24Configure fault-injection capabilities behavior
25-----------------------------------------------
26
27o debugfs entries
28
29fault-inject-debugfs kernel module provides some debugfs entries for runtime
30configuration of fault-injection capabilities.
31
156f5a78 32- /sys/kernel/debug/fail*/probability:
de1ba09b
AM
33
34 likelihood of failure injection, in percent.
35 Format: <percent>
36
5d0ffa2b
DM
37 Note that one-failure-per-hundred is a very high error rate
38 for some testcases. Consider setting probability=100 and configure
156f5a78 39 /sys/kernel/debug/fail*/interval for such testcases.
de1ba09b 40
156f5a78 41- /sys/kernel/debug/fail*/interval:
de1ba09b
AM
42
43 specifies the interval between failures, for calls to
44 should_fail() that pass all the other tests.
45
46 Note that if you enable this, by setting interval>1, you will
47 probably want to set probability=100.
48
156f5a78 49- /sys/kernel/debug/fail*/times:
de1ba09b
AM
50
51 specifies how many times failures may happen at most.
52 A value of -1 means "no limit".
53
156f5a78 54- /sys/kernel/debug/fail*/space:
de1ba09b
AM
55
56 specifies an initial resource "budget", decremented by "size"
57 on each call to should_fail(,size). Failure injection is
58 suppressed until "space" reaches zero.
59
156f5a78 60- /sys/kernel/debug/fail*/verbose
de1ba09b
AM
61
62 Format: { 0 | 1 | 2 }
5d0ffa2b
DM
63 specifies the verbosity of the messages when failure is
64 injected. '0' means no messages; '1' will print only a single
65 log line per failure; '2' will print a call trace too -- useful
66 to debug the problems revealed by fault injection.
de1ba09b 67
156f5a78 68- /sys/kernel/debug/fail*/task-filter:
de1ba09b 69
5d0ffa2b
DM
70 Format: { 'Y' | 'N' }
71 A value of 'N' disables filtering by process (default).
de1ba09b
AM
72 Any positive value limits failures to only processes indicated by
73 /proc/<pid>/make-it-fail==1.
74
156f5a78
GL
75- /sys/kernel/debug/fail*/require-start:
76- /sys/kernel/debug/fail*/require-end:
77- /sys/kernel/debug/fail*/reject-start:
78- /sys/kernel/debug/fail*/reject-end:
de1ba09b
AM
79
80 specifies the range of virtual addresses tested during
81 stacktrace walking. Failure is injected only if some caller
329409ae
AM
82 in the walked stacktrace lies within the required range, and
83 none lies within the rejected range.
84 Default required range is [0,ULONG_MAX) (whole of virtual address space).
85 Default rejected range is [0,0).
de1ba09b 86
156f5a78 87- /sys/kernel/debug/fail*/stacktrace-depth:
de1ba09b
AM
88
89 specifies the maximum stacktrace depth walked during search
5d0ffa2b
DM
90 for a caller within [require-start,require-end) OR
91 [reject-start,reject-end).
de1ba09b 92
156f5a78 93- /sys/kernel/debug/fail_page_alloc/ignore-gfp-highmem:
de1ba09b 94
5d0ffa2b
DM
95 Format: { 'Y' | 'N' }
96 default is 'N', setting it to 'Y' won't inject failures into
de1ba09b
AM
97 highmem/user allocations.
98
156f5a78
GL
99- /sys/kernel/debug/failslab/ignore-gfp-wait:
100- /sys/kernel/debug/fail_page_alloc/ignore-gfp-wait:
de1ba09b 101
5d0ffa2b
DM
102 Format: { 'Y' | 'N' }
103 default is 'N', setting it to 'Y' will inject failures
de1ba09b
AM
104 only into non-sleep allocations (GFP_ATOMIC allocations).
105
156f5a78 106- /sys/kernel/debug/fail_page_alloc/min-order:
54114994
AM
107
108 specifies the minimum page allocation order to be injected
109 failures.
110
de1ba09b
AM
111o Boot option
112
113In order to inject faults while debugfs is not available (early boot time),
114use the boot option:
115
116 failslab=
117 fail_page_alloc=
118 fail_make_request=<interval>,<probability>,<space>,<times>
119
120How to add new fault injection capability
121-----------------------------------------
122
123o #include <linux/fault-inject.h>
124
125o define the fault attributes
126
127 DECLARE_FAULT_INJECTION(name);
128
129 Please see the definition of struct fault_attr in fault-inject.h
130 for details.
131
5d0ffa2b 132o provide a way to configure fault attributes
de1ba09b
AM
133
134- boot option
135
136 If you need to enable the fault injection capability from boot time, you can
5d0ffa2b 137 provide boot option to configure it. There is a helper function for it:
de1ba09b 138
5d0ffa2b 139 setup_fault_attr(attr, str);
de1ba09b
AM
140
141- debugfs entries
142
143 failslab, fail_page_alloc, and fail_make_request use this way.
5d0ffa2b 144 Helper functions:
de1ba09b 145
a225a5cc
AB
146 init_fault_attr_dentries(entries, attr, name);
147 void cleanup_fault_attr_dentries(entries);
de1ba09b
AM
148
149- module parameters
150
151 If the scope of the fault injection capability is limited to a
152 single kernel module, it is better to provide module parameters to
153 configure the fault attributes.
154
155o add a hook to insert failures
156
5d0ffa2b 157 Upon should_fail() returning true, client code should inject a failure.
de1ba09b 158
5d0ffa2b 159 should_fail(attr, size);
de1ba09b
AM
160
161Application Examples
162--------------------
163
18584870 164o Inject slab allocation failures into module init/exit code
de1ba09b 165
de1ba09b
AM
166#!/bin/bash
167
18584870 168FAILTYPE=failslab
156f5a78
GL
169echo Y > /sys/kernel/debug/$FAILTYPE/task-filter
170echo 10 > /sys/kernel/debug/$FAILTYPE/probability
171echo 100 > /sys/kernel/debug/$FAILTYPE/interval
172echo -1 > /sys/kernel/debug/$FAILTYPE/times
173echo 0 > /sys/kernel/debug/$FAILTYPE/space
174echo 2 > /sys/kernel/debug/$FAILTYPE/verbose
175echo 1 > /sys/kernel/debug/$FAILTYPE/ignore-gfp-wait
de1ba09b 176
18584870 177faulty_system()
de1ba09b 178{
18584870 179 bash -c "echo 1 > /proc/self/make-it-fail && exec $*"
de1ba09b
AM
180}
181
18584870
AM
182if [ $# -eq 0 ]
183then
184 echo "Usage: $0 modulename [ modulename ... ]"
185 exit 1
186fi
187
188for m in $*
189do
190 echo inserting $m...
191 faulty_system modprobe $m
de1ba09b 192
18584870
AM
193 echo removing $m...
194 faulty_system modprobe -r $m
195done
de1ba09b
AM
196
197------------------------------------------------------------------------------
198
18584870 199o Inject page allocation failures only for a specific module
de1ba09b 200
de1ba09b
AM
201#!/bin/bash
202
18584870
AM
203FAILTYPE=fail_page_alloc
204module=$1
de1ba09b 205
18584870
AM
206if [ -z $module ]
207then
208 echo "Usage: $0 <modulename>"
209 exit 1
210fi
de1ba09b 211
18584870 212modprobe $module
de1ba09b 213
18584870
AM
214if [ ! -d /sys/module/$module/sections ]
215then
216 echo Module $module is not loaded
217 exit 1
218fi
219
156f5a78
GL
220cat /sys/module/$module/sections/.text > /sys/kernel/debug/$FAILTYPE/require-start
221cat /sys/module/$module/sections/.data > /sys/kernel/debug/$FAILTYPE/require-end
18584870 222
156f5a78
GL
223echo N > /sys/kernel/debug/$FAILTYPE/task-filter
224echo 10 > /sys/kernel/debug/$FAILTYPE/probability
225echo 100 > /sys/kernel/debug/$FAILTYPE/interval
226echo -1 > /sys/kernel/debug/$FAILTYPE/times
227echo 0 > /sys/kernel/debug/$FAILTYPE/space
228echo 2 > /sys/kernel/debug/$FAILTYPE/verbose
229echo 1 > /sys/kernel/debug/$FAILTYPE/ignore-gfp-wait
230echo 1 > /sys/kernel/debug/$FAILTYPE/ignore-gfp-highmem
231echo 10 > /sys/kernel/debug/$FAILTYPE/stacktrace-depth
18584870 232
156f5a78 233trap "echo 0 > /sys/kernel/debug/$FAILTYPE/probability" SIGINT SIGTERM EXIT
18584870
AM
234
235echo "Injecting errors into the module $module... (interrupt to stop)"
236sleep 1000000
de1ba09b 237