Uploaded image for project: 'Solution Center'
  1. Solution Center
  2. SOL-33

Backup-Monitoring

    XMLWordPrintable

    Details

    • Explanation:
      Hide

      The monitoring of backups is possible over EXAoperation logservice messages and the retrieval of backup information (e.g. via XML-RPC).

      In a logservice configured with a database and a minimum log priority of "Notice", the appropriate database will issue backup messages like

      • Backup started
      • Backup finished
      • Backup failed

      These messages can be retrieved via the XML-RPC functions logEntries() and logEntriesTagged() as well as in the EXAoperation Monitoring form.

      Possible backup failure reasons include, but are not limited to

      • low disk space
      • missing base backup for an incremental database backup
      • connection failures to remote backup servers

      When a backup has been finished, it will be listed in the result of the getBackupList() XML-RPC function. Further information can be retrieved via getBackupInfo().

      Here are some Python examples utilizing the standard xmlrpclib module for XML-RPC client functionality:

      >>> # Start a level-0 backup on archive volume v0000 for database exa_db1 with an expiration time of three days.
      >>> import xmlrpclib, pprint
      >>> s = xmlrpclib.ServerProxy("http://user:password@license-server/cluster1/db_exa_db1")
      >>> s.startStorageBackup('v0000', 0, '3d')
      
      [...]
      >>> # Get information about backups of the same database
      >>> s.getBackupList()
      [['2012-10-04 10:29', 1], ['2012-10-04 10:29', 2], ['2012-10-04 10:30', 3], ['2012-10-04 10:30', 4]]
      >>> # Now, we have a tuple of timestamps and IDs for each backup. We will use the timestamp for further actions.
      
      >>> pprint.pprint(s.getBackupInfo('2012-10-04 10:29'))
      {'comment': '',
      'dependencies': [],
      'expire date': '2012-10-07 10:29',
      'files': ['exa_db1/id_1/level_0/node_2/backup_201210041029',
      'exa_db1/id_1/level_0/node_1/backup_201210041029',
      'exa_db1/id_1/level_0/node_0/backup_201210041029',
      'exa_db1/id_1/level_0/node_0/metadata_201210041029'],
      'id': 1,
      'level': 0,
      'nodes': 3,
      'size': 150.07,
      'system': 'exa_db1',
      'timestamp': '2012-10-04 10:29',
      'usable': True,
      'usage': 100.3,
      'volume': ['v0000', 0]}
      

      Furthermore, here is an example for fetching new logentries with help of logEntriesTagged() (which can be used to fetch messages about the progress of backups - see above):

      >>> s = xmlrpclib.ServerProxy("http://user:password@license-server/cluster1/logservice1")
      >>> pprint.pprint(s.logEntriesTagged('my client tag'))
      [[2012, 10, 4, 10, 59, 47, 0],
      [2012, 10, 4, 11, 0, 47, 0],
      [{'message': 'n0011.c0001.exacluster.local 0.54 0.17
      'node': '1',
      'priority': 'Information',
      'strtime': '2012-10-04 11:00:01.780071+02:00',
      'system': 'load',
      'timestamp': '2012-10-04 11:00:01.780071'},
      {'message': 'n0013.c0001.exacluster.local 0.08 0.09
      'node': '4',
      'priority': 'Information',
      'strtime': '2012-10-04 11:00:01.700149+02:00',
      'system': 'load',
      'timestamp': '2012-10-04 11:00:01.700149'},
      ]]
      

      The EXAoperation manual contains even more programming examples in Python and Perl for connecting monitoring systems like Nagios to EXASuite clusters.

      Show
      The monitoring of backups is possible over EXAoperation logservice messages and the retrieval of backup information (e.g. via XML-RPC). In a logservice configured with a database and a minimum log priority of "Notice", the appropriate database will issue backup messages like Backup started Backup finished Backup failed These messages can be retrieved via the XML-RPC functions logEntries() and logEntriesTagged() as well as in the EXAoperation Monitoring form. Possible backup failure reasons include, but are not limited to low disk space missing base backup for an incremental database backup connection failures to remote backup servers When a backup has been finished, it will be listed in the result of the getBackupList() XML-RPC function. Further information can be retrieved via getBackupInfo(). Here are some Python examples utilizing the standard xmlrpclib module for XML-RPC client functionality: >>> # Start a level-0 backup on archive volume v0000 for database exa_db1 with an expiration time of three days. >>> import xmlrpclib, pprint >>> s = xmlrpclib.ServerProxy( "http: //user:password@license-server/cluster1/db_exa_db1" ) >>> s.startStorageBackup( 'v0000' , 0, '3d' ) [...] >>> # Get information about backups of the same database >>> s.getBackupList() [[ '2012-10-04 10:29' , 1], [ '2012-10-04 10:29' , 2], [ '2012-10-04 10:30' , 3], [ '2012-10-04 10:30' , 4]] >>> # Now, we have a tuple of timestamps and IDs for each backup. We will use the timestamp for further actions. >>> pprint.pprint(s.getBackupInfo( '2012-10-04 10:29' )) { 'comment' : '', 'dependencies' : [], 'expire date' : '2012-10-07 10:29' , 'files' : [ 'exa_db1/id_1/level_0/node_2/backup_201210041029' , 'exa_db1/id_1/level_0/node_1/backup_201210041029' , 'exa_db1/id_1/level_0/node_0/backup_201210041029' , 'exa_db1/id_1/level_0/node_0/metadata_201210041029' ], 'id' : 1, 'level' : 0, 'nodes' : 3, 'size' : 150.07, 'system' : 'exa_db1' , 'timestamp' : '2012-10-04 10:29' , 'usable' : True, 'usage' : 100.3, 'volume' : [ 'v0000' , 0]} Furthermore, here is an example for fetching new logentries with help of logEntriesTagged() (which can be used to fetch messages about the progress of backups - see above): >>> s = xmlrpclib.ServerProxy( "http: //user:password@license-server/cluster1/logservice1" ) >>> pprint.pprint(s.logEntriesTagged( 'my client tag' )) [[2012, 10, 4, 10, 59, 47, 0], [2012, 10, 4, 11, 0, 47, 0], [{ 'message' : 'n0011.c0001.exacluster.local 0.54 0.17 'node' : '1' , 'priority' : 'Information' , 'strtime' : '2012-10-04 11:00:01.780071+02:00' , 'system' : 'load' , 'timestamp' : '2012-10-04 11:00:01.780071' }, { 'message' : 'n0013.c0001.exacluster.local 0.08 0.09 'node' : '4' , 'priority' : 'Information' , 'strtime' : '2012-10-04 11:00:01.700149+02:00' , 'system' : 'load' , 'timestamp' : '2012-10-04 11:00:01.700149' }, ]] The EXAoperation manual contains even more programming examples in Python and Perl for connecting monitoring systems like Nagios to EXASuite clusters.
    • Category 1:
      Cluster Administration - Backup
    • Category 2:
      Cluster Administration - Monitoring & Logging

      Attachments

        Issue Links

          Activity

            People

            • Assignee:
              CaptainEXA Captain EXASOL
              Reporter:
              CaptainEXA Captain EXASOL
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: