====== Using INCLude/EXCLude Rules======

===== Notice=====

For creating In- and Exclude rules the TSM documentation at the [[http://pic.dhe.ibm.com/infocenter/tsminfo/v7r1/topic/com.ibm.itsm.client.doc/t_cfg_crtinclexcl.html|TSM-Infocenter]]) is suitable.
Therefore, only the most important statement is summarized.

**NOTICE:**
These examples are for Unices, but the statements are somehow identical on Windows. (''/'' => ''\'')
The configuration therefore will be in "dsm.opt" instead of "dsm.sys".

<WRAP center round important 90%>
The configuration of Include, but especially **''EXCLUDE''** rules requires the highest attention as wrongly placed or wrong defined rules will lead to files excluded from backup! Subsequent changes may discard already existing backups!

**//The GWDG//** therefore can only give suggestions and recommendations for creating rules, **//but is not taking any responsibility for the correctness of examples//**!

Please verify your chosen rules before use, either by a backup with the option //preview// in the GUI or calling //dsmc backup preview//!
</WRAP>

===== Using Wildcards and Special Chrarcters =====

TSM allows the usage of Wildcards in In- and Exclude rules:
  * ''?'' representing exactly one random character
  * ''*'' representing any number of random characters

  * ''/.../'' representing any number of directory levels
<code>/home/.../public</code>
Representing all directories ''public'' within ''/home'', independent of the directory level
<code>/srv/.../*.zip</code>
Is representing all ''.zip'' files within ''/srv''.

  * Groups of characters can be summed as a regular expression : ''[<list of characters>]''. You can use a dash to enter ranges. Example: <code>exclude /Volumes/Vol[1-4]/.../*.cpp</code> is selecting all files with the extension ''.cpp'' from the directories ''Vol1'',''Vol2'', ''Vol3'' and ''Vol4'' within ''/Volumes''. <code>exclude /home/[a-c,A-C]*/.../*.cpp</code> excludes all files from home directory starting with an ''a'',''b'',''c'',''A'',''B'' or ''C''.


If a file or directory name includes special characters, they have to be masked 
  * Usually whitespaces can be captured in quotation marks:<code>"My Files"</code>
  * The special characters <code> * ? : [ ]</code> have to be masked by a <code> [\<Symbol]</code> . To exclude the file <code>Text[2].txt</code> following rule needs to be set:<code>exclude /.../Text[\[]2[\]]</code>.
  * To Exclude the directory <code>/usr/src/classes/project::tmp</code> following rule needs to be set:<code>exclude.dir  /usr/src/classes/project[\:][\:]tmp</code>

===== Outsource Include/Exclude Rules =====

In- and exclude rules can get extensiv fast and therefore will substantially affect the configuration readability.
In this state or if you want to group rules you may find it helpful to outsource these rules in another or multiple other files.
These files can be imported into the "dsm.sys" using the parameter ''INCLExcl <Filename>''.
Example:
  INCLExcl  inclexcl.user
  INCLExcl  inclexcl.linux
  INCLExcl  inclexcl.gwdg


<WRAP center round important 90%>
Using ''INCLExcl'' to import a non existing file an error will occur. Please validate "dsm.sys" by testing.
</WRAP>

=====  Hierarchy of InExClude =====
 
  * Generally the TSM client works through the ''dsm.sys'' file **last to first**.
  * The **rules which match first will be applied**, rules matching later may won't affect (See Example1).
<WRAP center round tip 90%>
=== Example1: ===

  include /Users/user01/Documents/includefile.cpp
  exclude /Users/user01/Documents/.../*
=> The file ''includefile.cpp'' **will not be backed up** caused by the previous exclusion of the directory ''/Users/user01/Documents''.

=== Example2: ===

  exclude /Users/user01/Documents/.../*
  include /Users/user01/Documents/includefile.cpp
=> The file ''includefile.cpp'' **will be backed up**, since it was explicitly included by using ''include'' before the exclusion of the entire directory ''/Users/user01/Documents''.
</WRAP>


  * Rules which exclude entire //filespaces// (''exclude**.fs**'') or //directories// (''exclude**.dir**'') will be applied before all other rules.
  * Rules in a separate file will be applied at that time where the ''INEXClude <file>'' is placed in the ''dsm.sys''.
===== Management Classes =====

The Management Class on the TSM server, which should be used for the backup can be managed also by the include rules:
  include <path or file | search pattern> <Management Class>

This feature applies only in individual cases at the backup service by GWDG.
If so, we will contact you personally.

===== Compression =====

All data will be compressed due to sever side default settings which enable the data transfer compression.
Compressed data (e.g. ''.zip'',''.gz'' as well as ''.jpg'', etc.) will be tried to be compressed again.
This will most likely result in a high computational load rather than result in data reduction.

To avoid this unnecessary load all compressed data should be excluded from compression.
The relevant rules should be applied to the end of the file ''dsm.sys''(Unices) or ''dsm.opt''(Windows).
(See [[en:services:storage_services:backup:tsm:inclexcl#hierarchy_of_inexclude|Hierarchie]] above)

//Suggestion for exclusion of compressed files://
  exclude.compression     /.../*.z
  exclude.compression     /.../*.Z
  exclude.compression     /.../*.gz
  exclude.compression     /.../*.GZ
  exclude.compression     /.../*.bz
  exclude.compression     /.../*.BZ
  exclude.compression     /.../*.bz2
  exclude.compression     /.../*.BZ2
  exclude.compression     /.../*.tgz
  exclude.compression     /.../*.TGZ
  exclude.compression     /.../*.gif
  exclude.compression     /.../*.GIF
  exclude.compression     /.../*.jpg
  exclude.compression     /.../*.JPG
  exclude.compression     /.../*.mpg
  exclude.compression     /.../*.MPG
  exclude.compression     /.../*.mp4
  exclude.compression     /.../*.MP4
  exclude.compression     /.../*.wmv
  exclude.compression     /.../*.WMV
  exclude.compression     /.../*.mp3
  exclude.compression     /.../*.MP3
  exclude.compression     /.../*.aac
  exclude.compression     /.../*.AAC
  exclude.compression     /.../*.wma
  exclude.compression     /.../*.WMA
  exclude.compression     /.../*.zip
  exclude.compression     /.../*.ZIP
  exclude.compression     /.../*.rar
  exclude.compression     /.../*.RAR
  exclude.compression     /.../*.pdf
  exclude.compression     /.../*.PDF
  exclude.compression     /.../*.cdr
  exclude.compression     /.../*.CDR
  exclude.compression     /.../*.ppt
  exclude.compression     /.../*.PPT
  exclude.compression     /.../*.pst
  exclude.compression     /.../*.PST
  exclude.compression     /.../*.rpm
  exclude.compression     /.../*.deb
 \\
===== Encryption =====

TSM supports file encryption before sending the data to a server. Beside the indisputable advantages the encryption comes with some disadvantages as well:
  * The encryption is preventing every access to the data. Even from the TSM administrator.
  * There is no way to decrypt the data if the password is lost!
Some disadvantages can be neglected:
  * The compression rate is usually low.
  * Functions as //subfile backup// or //client side deduplictaion// and //server side deduplication// can't be used with encryption -- both are not offered by GWDG in context of the file back up anyway.

Encryption can be used by defining rules for the files which should be encrypted. Therefore, you have to use the keyword //encrypt// with an include statement, e.g.:

  include.encrypt /home/.../*
  exclude.encrypt /home/.../public/.../*

\\
===== Excluding TSM files =====

Backing up the TSM directories is in general not very useful.
Therefore, it could be excluded by ''exclude.dir'' rule.
Somehow, if the configuration files should be backed up, two approaches are possible:
  - The files could be saved at a separate path and only have a symlink / shortcut in the TSM client directory. The TSM directory is excluded as mentioned above.
  - Using wildcards instead of ''exclude.dir'' to exclude the TSM directory. The explicit include for the TSM configuration files must then be set:

  exclude /opt/tivoli/tsm/.../*
  include /opt/tivoli/tsm/client/ba/bin/dsm.sys
  include /opt/tivoli/tsm/client/ba/bin/dsm.opt
  include /opt/tivoli/tsm/client/ba/bin/tsmjbbd.ini
\\
===== Server Side Include-Exclude Rules =====

Server sided in- exclude rules are overriding the client side configuration.
The GWDG largely dispenses server sided rules 
since only the administrator can usefully define the files to be excluded or encrypted.


At the present time (date of this text) only the management class //3COPIES// is specified to back up the //system state// (Windows only) to limit the amount of //system state// versions by the number of three.
Relevant only for //Bare Metal// restores, the //system state// uses an excessive amount of storage.  
The system restore points in //Windows// providing an easier way for restoring older system states on running systems.