This is a pass-through script for SNMP that gives all the statistics for all finished Bacula (http://www.bacula.org/) jobs.
This SNMP/perlscript is indexed, to makes it easier to get information into Cacti (//www.raxnet.net/products/cacti/).
It is currently in soon-to-be finished state (laying the final touches of the script - recursion in Perl is difficult ! :).
Now most of the traversing works fine, all the values are correct ones (even though all the values that are bigger than an Integer32 (2^32-1) is shown with an error. This is due to a 'missing feature' (which can't be rectified because of RFC's and standards etc). This lead to the tables being shown slightly incorrect.
Also, the stats table isn't finnished quite yet. It depends (includes indexes from) on three other tables (which IS shown correctly). I'm not quite sure how to solve this, but I'm working on it...
----- s n i p -----
root@aurora:~# /usr/bin/snmptable -Ci -Cb localhost public .1.3.6.1.4.1.8767.2.3.5
SNMP table: enterprises.bayourCom.snmp.baculaStats.baculaClientsTable
index sIndex Name Uname AutoPrune RetentionFile RetentionJob
1 1 aurora.bayour.com-fd sparc-unknown-linux-gnu,debian,3.0 true 1181 2531
2 2 fritz.bayour.com-fd sparc-unknown-linux-gnu,debian,3.0 true 1181 2531
3 3 pumba.bayour.com-fd i386-pc-linux-gnu,debian,3.0 true 1181 2531
4 4 rmgztk.bayour.com-fd sparc-unknown-linux-gnu,debian,3.0 true 1181 2531
---------------------------------------------------------------------------------- root@aurora:~ /usr/bin/snmptable -Ci -Cb localhost public .1.3.6.1.4.1.8767.2.3.6 SNMP table: enterprises.bayourCom.snmp.baculaStats.baculaJobsNameTable
index NameIndex Name Desc MD5 Create 1.1 1 Aurora_System Aurora System 65tqLkAGM9pzCn/nE/+oTD 2005-04-18 10:23:04 1.2 2 Aurora_Web 1.3 3 Aurora_Web_Stats Web stats directory B+gyED+b8XAC7Wwdfn+TXC 2005-04-18 05:00:48 1.4 4 Backup_Catalog Catalog Ai/lZ4xS05/be4c8+++eyD 2005-04-17 17:30:51 1.5 5 Backup_Devel Old_Devel Installation AH4x8G/99//4BU/zd+sBxC 2005-06-08 01:10:04 1.6 6 Backup_Homes Volume backups JzkYK7pIuHc7jE+x4w+j1A 2005-09-29 04:21:56 1.7 7 Backup_Mail Mail directory 9/oGR0//N2gWK+9/xV+hFA 2005-09-28 01:09:03 2.1 1 Fritz_System Fritz System VA+0qn+cW/xohjQgS68gtC 2005-06-18 01:35:05 3.1 1 Pumba_System Pumba System I4+5k6+v47+9Y6/9CCpSaC 2005-04-18 08:41:55 4.1 1 Rmgztk_System
---------------------------------------------------------------------------------- root@aurora:~ /usr/bin/snmptable -Ci -Cb localhost public .1.3.6.1.4.1.8767.2.3.7 SNMP table: enterprises.bayourCom.snmp.baculaStats.baculaJobsIDTable
index Index
1.1.1 1 Aurora_System.2005-09-23_03.00.01 1.1.2 2 Aurora_System.2005-10-13_03.00.01 1.1.3 3 Aurora_System.2005-10-14_03.00.01 1.1.4 4 Aurora_System.2005-10-15_03.00.01 1.2.1 1 Aurora_Web.2005-09-24_01.05.00 1.2.2 2 Aurora_Web.2005-09-25_01.05.01 1.2.3 3 Aurora_Web.2005-10-13_01.05.01 1.2.4 4 Aurora_Web.2005-10-14_01.05.01 1.2.5 5 Aurora_Web.2005-10-15_01.05.00 1.3.1 1 Aurora_Web_Stats.2005-10-13_01.05.02 1.3.2 2 Aurora_Web_Stats.2005-10-14_01.05.02 1.3.3 3 Aurora_Web_Stats.2005-10-15_01.05.01 1.4.1 1 Backup_Catalog.2005-09-17_03.10.00 1.4.2 2 Backup_Catalog.2005-09-23_03.10.00 1.4.3 3 Backup_Catalog.2005-10-14_03.10.00 1.5.1 1 Backup_Devel.2005-10-13_01.05.04 1.5.2 2 Backup_Devel.2005-10-14_01.05.04 1.5.3 3 Backup_Devel.2005-10-15_01.05.03 1.6.1 1 Backup_Homes.2005-10-13_01.05.07 1.6.2 2 Backup_Homes.2005-10-14_01.05.07 1.6.3 3 Backup_Homes.2005-10-15_01.05.06 1.7.1 1 Backup_Mail.2005-10-13_01.05.03 1.7.2 2 Backup_Mail.2005-10-14_01.05.03 1.7.3 3 Backup_Mail.2005-10-15_01.05.02 2.1.1 1 Fritz_System.2005-10-13_01.05.06 2.1.2 2 Fritz_System.2005-10-14_01.05.06 2.1.3 3 Fritz_System.2005-10-15_01.05.05 3.1.1 1 Pumba_System.2005-10-13_01.05.05 3.1.2 2 Pumba_System.2005-10-14_01.05.05 3.1.3 3 Pumba_System.2005-10-15_01.05.04 4.1.1 1 Rmgztk_System.2005-09-20_01.05.06
---------------------------------------------------------------------------------- root@aurora:~ /usr/bin/snmptable -Ci -Cb localhost public .1.3.6.1.4.1.8767.2.3.8 SNMP table: enterprises.bayourCom.snmp.baculaStats.baculaStatsCountersTable
index sIndex Name
1 1 start_date
2 2 end_date
3 3 duration
4 4 files
5 5 bytes
6 6 type
7 7 status
8 8 level
9 9 errors
10 10 missing
---------------------------------------------------------------------------------- root@aurora:~ /usr/bin/snmptable -Ci localhost public .1.3.6.1.4.1.8767.2.3.9
index baculaStatsIndex baculaStatsStart baculaStatsEnd baculaStatsDuration baculaStatsFiles baculaStatsBytes baculaStatsType baculaStatsStatus baculaStatsLevel baculaStatsErrors baculaStatsMissingFiles 1.1.1 1 2005-09-23 13:06:48 2005-09-23 13:16:03 555 0 0 B 0 d 0 0 1.1.2 2 2005-10-13 14:54:47 2005-10-13 16:08:38 4431 2003 4177858 B 0 d 0 0 1.1.3 3 2005-10-15 19:54:30 2005-10-15 19:54:30 0 0 0 B 0 d 0 0 1.1.4 4 2005-10-15 20:27:30 2005-10-15 20:27:30 0 0 0 B 0 d 0 0 1.2.1 ? 2005-09-24 01:05:01 0000-00-00 00:00:00 -1 0 0 B 0 d 0 0 1.2.2 2 2005-09-25 01:05:02 0000-00-00 00:00:00 -1 0 0 B 0 d 0 0 1.2.3 3 2005-10-13 01:05:02 2005-10-13 01:21:25 983 6625 791533 B 0 d 0 0 1.2.4 4 2005-10-14 01:05:02 2005-10-14 01:05:11 9 11 244 B 0 d 0 0 1.2.5 5 2005-10-15 20:58:30 2005-10-15 20:58:30 0 0 0 B 0 d 0 0 1.2.6 6 2005-10-22 01:05:01 0000-00-00 00:00:00 -1 0 0 B 0 d 0 0 1.3.1 ? 2005-10-13 01:30:47 2005-10-13 01:31:29 42 2122 6995 B 0 d 0 0 1.3.2 2 2005-10-14 01:06:06 2005-10-14 01:06:14 8 0 0 B 0 d 0 0 1.3.3 3 2005-10-15 21:28:30 2005-10-15 21:28:30 0 0 0 B 0 d 0 0 1.4.1 ? 2005-09-17 09:29:34 2005-09-17 09:33:38 244 132 20836 B 0 d 0 0 1.4.2 2 2005-09-23 13:16:04 2005-09-23 13:48:11 1927 0 0 B 0 d 0 0 1.4.3 3 2005-10-15 11:31:30 2005-10-15 11:31:30 0 0 0 B 0 d 0 0 1.5.1 ? 2005-10-13 01:38:15 2005-10-13 01:38:18 3 0 0 B 0 d 0 0 1.5.2 2 2005-10-14 01:08:33 2005-10-14 01:08:36 3 0 0 B 0 d 0 0 1.5.3 3 2005-10-15 22:28:30 2005-10-15 22:28:30 0 0 0 B 0 d 0 0 1.6.1 ? 2005-10-13 02:12:28 2005-10-13 14:54:46 45738 35 20706161 B 0 d 0 0 1.6.2 2 2005-10-14 01:15:23 2005-10-14 13:15:56 43233 31 16824497 B 0 d 0 0 1.6.3 3 2005-10-15 16:05:29 2005-10-15 16:05:29 0 0 0 B 0 d 0 0 1.7.1 ? 2005-10-13 01:31:30 2005-10-13 01:38:14 404 21281 61597 B 0 d 0 0 1.7.2 2 2005-10-14 01:06:18 2005-10-14 01:08:33 135 1933 9281 B 0 d 0 0 1.7.3 3 2005-10-15 21:58:30 2005-10-15 21:58:30 0 0 0 B 0 d 0 0 2.1.1 ? 2005-10-13 01:48:00 2005-10-13 02:11:09 1389 14487 1030864 B 0 d 0 0 2.1.2 2 2005-10-14 01:12:20 2005-10-14 01:15:22 182 1305 125810 B 0 d 0 0 2.1.3 3 2005-10-15 23:28:30 2005-10-15 23:28:30 0 0 0 B 0 d 0 0 3.1.1 ? 2005-10-13 01:38:19 2005-10-13 01:46:40 501 2261 589233 B 0 d 0 0 3.1.2 2 2005-10-14 01:08:37 2005-10-14 01:12:19 222 1568 324314 B 0 d 0 0 3.1.3 3 2005-10-15 22:58:30 2005-10-15 22:58:30 0 0 0 B 0 d 0 0 4.1.1 ? 2005-09-20 12:01:47 0000-00-00 00:00:00 -1 0 0 B 0 d 0 0
root@aurora:~# /usr/bin/snmptable -Ci -Cb localhost public .1.3.6.1.4.1.8767.2.3.10 SNMP table: enterprises.bayourCom.snmp.baculaStats.baculaPoolsTable
index Index Name Num Max UseOnce UseCatalog AcceptAnyVolume Retention Duration MaxJobs MaxFiles MaxBytes AutoPrune Recycle Type LabelFormat Enabled ScratchPoolID RecyclePoolID
1 1 Default 6 0 false true true 590 0 0 0 0 true true Backup * true 0 0
---------------------------------------------------------------------------------- root@aurora:~ /usr/bin/snmptable -Ci -Cb localhost public .1.3.6.1.4.1.8767.2.3.11 SNMP table: enterprises.bayourCom.snmp.baculaStats.baculaMediaTable
index Index Name Slot Pool Type WrittenFirst WrittenLast LabelDate Jobs Files Blocks Mounts Bytes Errors Writes Capacity Status Recycle Retention Duration MaxJobs MaxFiles MaxBytes InChanger MediaAddressing ReadTime WriteTime EndFile EndBlock
1 1 Tape_1a 0 1 Ultrium1 2005-09-27 01:05:02 2005-10-14 09:09:07 2005-09-27 01:05:04 62 201 2345616 12 147771503 0 4862225 0 0 true 590 0 0 0 0 true false 0 0 200 8989
2 2 Tape_2 0 1 Ultrium1 2005-07-28 01:05:02 2005-08-03 01:05:59 2005-07-28 09:31:33 32 87 971119 4 61179164 0 2618531 0 0 true 590 0 0 0 0 false false 0 0 87 11651
3 3 Tape_3 0 1 Ultrium1 2005-08-04 01:05:40 2005-08-15 04:24:38 2005-10-14 09:14:36 0 0 0 3 0 0 3299060 0 0 true 590 0 0 0 0 false false 0 0 170 2927
4 4 Tape_4 0 1 Ultrium1 2005-08-22 13:53:58 2005-09-10 01:59:34 2005-08-22 15:38:36 51 144 1649075 13 103889933 0 6750966 0 0 true 590 0 0 0 0 false false 0 0 143 2241
5 5 Tape_5 0 1 Ultrium1 2005-09-13 01:05:02 2005-09-23 04:29:54 2005-09-13 01:05:03 62 153 1652210 6 104086829 0 4957481 0 0 true 590 0 0 0 0 false false 0 0 152 8042
----- s n i p -----
Here you (might) see the problem I talked about above: The file sizes and baculaClientRetention{Job,File} etc is in KILO bytes, not BYTES as the SQL data is in. This is because I can't use a Counter64 (the only type in SMI that is big enough to hold the BYTE value). This because it (Counter64) isn't (and will not) be implemented in net-snmp (because the pass-through system originates from SMIv1 which didn't have Counter64). As it is now, the value is a Counter32 which is ok.
I've now managed to get the 'baculaStatsTable' table to work. Kind'a. There's still something fishy about the index values, but that's ok (for now).
NOTE: This is now fixed in the CVS version:
----- s n i p -----
root@aurora:~# /usr/bin/snmptable -Ci localhost public .1.3.6.1.4.1.8767.2.3.9
SNMP table: enterprises.bayourCom.snmp.baculaStats.baculaStatsTable
index baculaStatsIndex baculaStatsStart baculaStatsEnd baculaStatsDuration baculaStatsFiles baculaStatsBytes baculaStatsType baculaStatsStatus baculaStatsLevel baculaStatsErrors baculaStatsMissingFiles 1.1.1 1 2005-09-23 13:06:48 2005-09-23 13:16:03 555 0 0 B 0 d 0 0 1.1.2 2 2005-10-13 14:54:47 2005-10-13 16:08:38 4431 2003 4177858 B 0 d 0 0 1.1.3 3 2005-10-15 19:54:30 2005-10-15 19:54:30 0 0 0 B 0 d 0 0 1.1.4 4 2005-10-15 20:27:30 2005-10-15 20:27:30 0 0 0 B 0 d 0 0 1.2.1 1 2005-09-24 01:05:01 0000-00-00 00:00:00 -1 0 0 B 0 d 0 0 1.2.2 2 2005-09-25 01:05:02 0000-00-00 00:00:00 -1 0 0 B 0 d 0 0 1.2.3 3 2005-10-13 01:05:02 2005-10-13 01:21:25 983 6625 791533 B 0 d 0 0 1.2.4 4 2005-10-14 01:05:02 2005-10-14 01:05:11 9 11 244 B 0 d 0 0 1.2.5 5 2005-10-15 20:58:30 2005-10-15 20:58:30 0 0 0 B 0 d 0 0 1.2.6 6 2005-10-22 01:05:01 0000-00-00 00:00:00 -1 0 0 B 0 d 0 0 1.3.1 1 2005-10-13 01:30:47 2005-10-13 01:31:29 42 2122 6995 B 0 d 0 0 1.3.2 2 2005-10-14 01:06:06 2005-10-14 01:06:14 8 0 0 B 0 d 0 0 1.3.3 3 2005-10-15 21:28:30 2005-10-15 21:28:30 0 0 0 B 0 d 0 0 1.4.1 1 2005-09-17 09:29:34 2005-09-17 09:33:38 244 132 20836 B 0 d 0 0 1.4.2 2 2005-09-23 13:16:04 2005-09-23 13:48:11 1927 0 0 B 0 d 0 0 1.4.3 3 2005-10-15 11:31:30 2005-10-15 11:31:30 0 0 0 B 0 d 0 0 1.5.1 1 2005-10-13 01:38:15 2005-10-13 01:38:18 3 0 0 B 0 d 0 0 1.5.2 2 2005-10-14 01:08:33 2005-10-14 01:08:36 3 0 0 B 0 d 0 0 1.5.3 3 2005-10-15 22:28:30 2005-10-15 22:28:30 0 0 0 B 0 d 0 0 1.6.1 1 2005-10-13 02:12:28 2005-10-13 14:54:46 45738 35 20706161 B 0 d 0 0 1.6.2 2 2005-10-14 01:15:23 2005-10-14 13:15:56 43233 31 16824497 B 0 d 0 0 1.6.3 3 2005-10-15 16:05:29 2005-10-15 16:05:29 0 0 0 B 0 d 0 0 1.7.1 1 2005-10-13 01:31:30 2005-10-13 01:38:14 404 21281 61597 B 0 d 0 0 1.7.2 2 2005-10-14 01:06:18 2005-10-14 01:08:33 135 1933 9281 B 0 d 0 0 1.7.3 3 2005-10-15 21:58:30 2005-10-15 21:58:30 0 0 0 B 0 d 0 0 2.1.1 1 2005-10-13 01:48:00 2005-10-13 02:11:09 1389 14487 1030864 B 0 d 0 0 2.1.2 2 2005-10-14 01:12:20 2005-10-14 01:15:22 182 1305 125810 B 0 d 0 0 2.1.3 3 2005-10-15 23:28:30 2005-10-15 23:28:30 0 0 0 B 0 d 0 0 3.1.1 1 2005-10-13 01:38:19 2005-10-13 01:46:40 501 2261 589233 B 0 d 0 0 3.1.2 2 2005-10-14 01:08:37 2005-10-14 01:12:19 222 1568 324314 B 0 d 0 0 3.1.3 3 2005-10-15 22:58:30 2005-10-15 22:58:30 0 0 0 B 0 d 0 0 4.1.1 1 2005-09-20 12:01:47 0000-00-00 00:00:00 -1 0 0 B 0 d 0 0
----- s n i p -----
Now that the script is returning expected values and seems to work fine, I started looking at the Cacti XML template file. That didn't go TO well: I'm not quite sure about the format of this file. Since the stats table depends on three other tables - the clients, jobnames and jobIDs tables - it complicates matters. In the CVS version, I get the following when adding the 'SNMP - Local - Bacula Statistics - QUERY' SNMP query to a host:
----- s n i p -----
Index (clients) Client name Index (Job names) Job names Index (Job IDs) Job IDs Index (Job Status) Counter name
1 aurora.bayour.com-fd 1 Rmgztk_System 1 Rmgztk_System.2005-09-20_01.05.06 1 start_date
2 fritz.bayour.com-fd 2 Aurora_Web 2 Pumba_System.2005-10-14_01.05.05 2 end_date
3 pumba.bayour.com-fd 3 Aurora_Web_Stats 3 Pumba_System.2005-10-15_01.05.04 3 duration
4 rmgztk.bayour.com-fd 4 Backup_Catalog 4 Aurora_Web.2005-10-14_01.05.01 4 files
5 Backup_Devel 5 Aurora_Web.2005-10-15_01.05.00 5 bytes
6 Backup_Homes 6 Aurora_Web.2005-10-22_01.05.00 6 type
7 Backup_Mail 7 Aurora_Web.2005-10-23_01.05.01 7 status
level
errors
missing
----- s n i p -----
This is OBVIOSLY wrong, in more than one place! Working on a fix...
If you'd like to test (and help!) it out, you can run it like this (it will output EVERYTHING from the SQL server):
bacula-snmp-stats.pl -d -d -a
Also see the readme in the cvs directory (not this file) for more information on how I created the Cacti templates etc...
Current release is: 1.4
Tarball 1: http://www.bayour.com/bacula-snmp/bacula-snmp_1.4.tgz Tarball 2: http://www.bayour.com/bacula-snmp/bacula-snmp_1.4.tar.bz2 ZIPfile: http://www.bayour.com/bacula-snmp/bacula-snmp_1.4.zip Old tarballs: http://www.bayour.com/bacula-snmp/old/
SNMP Setup files (Paths depend on where your SNMP/Cacti is installed!)
BAYOUR-COM-MIB.txt This is the MIB declaration.
Copy to /usr/share/snmp/mibs/
BayourCOM_SNMP.pm This is the new Perl library with functions used in all my
SNMP modules.
Copy to a perl include directory. Find these with the command:
perl -e "foreach \$dir (@INC) { print \"\$dir\n\"; }" | sort
bacula-snmp-stats.pl This is the stat retreival script.
Copy to /etc/snmp/
snmp.conf.stub This is part of the snmp.conf file.
Add to the end of /etc/snmp/snmp.conf
snmpd.conf.stub This is part of the snmpd.conf file.
Add to the end of /etc/snmp/snmpd.conf
Cacti setup files
NOTE: These two isn't even remotely finnished, barely started.
There is absolutly no reason to load ANY of these (unless you'd
like to help writing them), because they will not give you any
data. I have to finish the perl script before there's any point
in dealing with this.
bacula-stats.xml This is the XML declaration for cacti.
Copy to /usr/share/cacti/resource/snmp_queries/
cacti_data_query_snmp_local_bacula_statistics_query.xml
This is all the templates to import into
cacti to retreive Bind9 statistics via
a Indexed SNMP query.
NOTE: This should be ok, but the final
stage (adding 'associated graph templates'
to the 'Data Queries' doesn't work. Probably
because the bacula-stats.xml isn't correct.
- NOTE
- The latest version of these files can be found via anonymous cvs (just press the ENTER key when asked for a password):
cvs -d :pserver:anonymous@cvs.bayour.com:/var/cvs login cvs -d :pserver:anonymous@cvs.bayour.com:/var/cvs co snmp/BayourCOM_SNMP.pm cvs -d :pserver:anonymous@cvs.bayour.com:/var/cvs co snmp/BAYOUR-COM-MIB.txt cvs -d :pserver:anonymous@cvs.bayour.com:/var/cvs co snmp/bacula