System Tap

System tap

System tap is a profiling tool that allows you to hook or probe live kernel and find out various information without recompilation of kernel.

Most important work you can do with system tap is to find out execution time of particular code block or function ,also you can print values passed to functions or local variables.

Following are short notes I have taken.These will provide you pointers to find out other relevant information on google :)

Stab -L 'kernel.trace("*")'	Return all possible probe points
/usr/share/systemtab/tabset	Look through tab sets
/usr/share/doc/<systemtab>/examples	Sample examples

Installation :

1.       Install -devel,-debuginfo,-debuginfo-common-arch

2.       yum install systemtap systemtap-runtime

Sample code :

stab -v -e `probe vfs.read { printf("read performed\n");exit()}`

Commonly used commands :

-v	Find error while running script
-o	Send output to filename
-S size,count	Limit size of log and count
-x PID	SystemTap handler function target() to PID
-c command	Set SystemTab handler target () to specific command ( use full path " stap script  -c /bin/cp " )
-e	Script string rather than file
-F	Make script background process

EVENTS :

These are the events we can possibly probe :

1.       Entering or leaving function

2.       Time expiration

3.       Session expiration

 

General syntax :

Probe event { stmt }

File extension

.stp

Functions :

function  function_name (arg) { stmt }

Probe event { function_name(args) }

EVENTS description :

Synchronous Events :

·         Process Executes and instruction

·   Examples

·   Syscall.system_call - enter in call

·   Syscall.close.return - return from call

·   Vfs.file_operation

·   Kernel.function("Function")

• Probe kernel.function("@net/socket.c").return

·   Kernel.trace("point")-specific trace point

• Kernel.trace("kfree_skb")

·   Module("ext3").fucntion("*").return

Asynchronous Event :

Not tied to particular instruction or location

Counters , timers etc.

Examples

·         Begin

·         End

·         Timer event

o    Probe timer.s/ms/us/hz/jiffies(number)

Example of timer

Probe timer.s(5)

{

p.f.("Hello');

}

Note : we can use * wild card character

Systemtap Body

probe begin

{

pf("hi\n")

exit()    //exit script or use ctrl+c

}

Functions we can use as argument

Tid() tread

Uid() user

Cpu()

Gettimeofday_s()-seconds since epoch

Ctime() -date

Pp()-prope point being habdled

Thread_indent()

Execname()

Pid()

probe syscall.* {	// all system calls
if (pid() == target())	// target() specify name supplied while running script
printf("%s/n", name)	// name prints system call name
}

HELP !!!

Man stapprobes	For probes
Man stapfuncs	For function

various construct of system tap script :

Variables

Conditional statements

If else

loop

While

For

Conditional operator

== , <= ,>= ,!=

Command line argument

$<arg num> - number

@<arg num > - string

Associative array

array_name[index_expression]

Examples :

foo["tom"] = 23

foo["dick"] = 24

foo["harry"] = 25

device[pid(),execname(),uid(),ppid(),"W"] = devname

NOTE : All associative array must be declared global

foo[tid()] = gettimeofday_s()

Sets time value to TID

Foreach examples to print array contains :

probe timer.s(3)
{
foreach (count in reads- limit 10)	Here - indicate print in descending order of values and limit Indicate number of 1st elements to print
printf("%s : %d \n", count, reads[count])
delete reads	Removes all elements from array
}

Check for member in array

if(["stapio"] in reads) {	Checks whether  stapio is in array
printf("stapio read detected, exiting\n")
exit()
}

Statistical aggregate

Add multiple values to same key / element

global reads
probe vfs.read
{
reads[execname()] <<< count	Add multiple values  of amount of data written
}

Extract data collected by statistical aggregate

Syntax :

@extractor(variable/array index expression).extractor

Extractor :

@count(writes[execname()])	Return number of elements
@sum(writes[execname()])	Return total
Min	Smallest among all
Max	Largest among all
Avg	Average of all

Tapsets

Pre-written probes and functions to be used

/usr/share/systemtap/tapset/

Examples :

/usr/share/systemtap/testsuite/systemtap/examples/