A take on tm1project

Table of contents

• A take on tm1project
• Overview
  • Git in the context of TM1
  • What is a tm1project?
  • Scope and limitations
• How could you use it?
  • Sample file
• Why should you consider it?
• Supported tm1project components in TM1Py

Overview

The idea of this post is to provide some guidance regarding TM1-Git functionality in the context of code promotion and how you could use it with new TM1Py support for tm1project. First, let’s go over the basics.

Git in the context of TM1

If you are a regular TM1 developer most likely you have never used Git before; but even if you did, you will soon realize that tm1-git is not really the same as git. Below is a list of key differences between git and tm1 flavour of git:

If you are unfamiliar with tm1-git, or you’ve never seen it in action, I would strongly suggest you to go over these 2 articles:

• GIT integration for TM1, part 1
• GIT integration for TM1, part 2

They provide step-by-step instructions on how you can set up and interact with a Git repository from scratch. Bear in mind that all the REST API functions shown there are now available in TM1Py (as of release 1.10)

What is a tm1project?

A tm1project is where you can define how a TM1 model should be published (pushed) and deployed (pulled). You can specify the objects that get published as well as the tasks that should be executed before, during and after a deployment. You can think of it as the place you set up your strategy towards code promotion. You can review IBM Documentation on tm1project for further details.

Scope and limitations

As mentioned above in the Git vs TM1-Git table, there are some limitations that you need to be aware of before trying to use this approach:

• understand that tm1-git (with or without a tm1project) is only intended publish the structure of a TM1 model, NOT the data inside it.
• tm1-git does not remove objects from the tm1server (even if they were deleted from the source branch), this is by design.
• physical dimension order is NOT part of the published cube object for TM1 v11 request
• tm1-git does NOT allow you to commit selected objects, anything that has changed since last commit will be automatically staged and committed to the repo on your next push. You need to remove the unwanted objects from the branch to which they were committed to.

How could you use it?

Considering the limitations that this functionality has you should probably establish some ground rules amongst your development team to properly work with it:

1. Use an ‘official’ tm1project (like the one below) so everybody knows what gets pushed/pulled and what happens before and after each action. If you change it everytime just to push what you want it will become unpredictable. Build around it, not in it.
2. Using TM1Py, create a command line program to allow a developer to:
   • Initialize and uninitialize a tm1 db against a Git repo.
   • Push code to a repo branch
   • Pull code from a repo branch
3. Create one ‘official’ branch per each environment/deployment (DEV, QA, UAT, PROD, SUPPORT, etc.)
4. Do NOT push changes directly to official branches, create new branches based on them, so you can curate ¹them and then raise a Pull Request to merge changes.

Sample file

Let’s assume we have the following TI’s in our TM1 application (you can find these functionalities in bedrock with different names):

• application_push_wrapper: it will export attributes and security cubes TO a flat file under git_source_files/data folder
• application_pull_wrapper: it will import attributes and security cubes FROM a flat file under git_source_files/data folder
• application_post_deployment: it will run processes from a ‘instructions.txt’ file
• application_save_data_all: it will execute a SaveDataAll()
• application_create_backup: it will create a hard backup of the TM1_db using a powershell script under git_source_files/scripts
• zTest*: all processes that are not ment or ready to be pushed start with ‘zTest’ or ‘WIP’

{
  "Version": "1.0",
  "Name": "sample_tm1project",
  "Files": 
  [
    "git_source_files/data/*.*",
    "git_source_files/scripts/*.*"
  ],
  "Ignore": 
  [
    "!Dimensions('}ElementAttributes_')",
    "!Dimensions('}ElementSecurity_')",
    "!Cubes('}ElementAttributes_')",
    "!Cubes('}ElementSecurity_')",
    "Processes('zTest')",
    "Processes('WIP')"
  ],
  "Tasks":
  {
    "export_before_push":
    {
      "Process": "Processes('application_push_wrapper')"
    },
    "import_after_pull":
    {
      "Process": "Processes('application_pull_wrapper')"
    },
    "save_data":
    {
      "Process": "Processes('application_save_data_all')"
    },
    "backup_instance":
    {
      "Process": "Processes('application_create_backup')"
    },
    "post_deployment_task":
    {
      "Process": "Processes('application_post_deployment')"
    }
  },
  "PrePush":
  [
    "Tasks('export_before_push')"
  ],
  "PrePull":
  [
    "Tasks('save_data')",
    "Tasks('backup_instance')"
  ],
  "PostPull":
  [
    "Tasks('import_after_pull')",
    "Tasks('post_deployment_task')"
  ],
  "Deployments": 
  {
      "DEV":
      {
          "Settings":
          {
              "Administration":
              {
                  "PerformanceMonitorOn": true,
                  "PerfMonActive": true,
                  "TopLog":
                  {
                      "Enable": true
                  }
              }
          }
      },
      "QA":
      {
          "Settings":
          {
              "Administration":
              {
                  "PerformanceMonitorOn": true,
                  "PerfMonActive": true,
                  "TopLog":
                  {
                      "Enable": true
                  }
              }
          }
      },
      "PROD":
      {
          "Settings":
          {
              "Administration":
              {
                  "PerformanceMonitorOn": false,
                  "PerfMonActive": false,
                  "TopLog":
                  {
                      "Enable": true
                  }
              }
          }
      }
   }
}

What would this tm1project do?

• It will push all files under the git_source_files/data AND git_source_files/scripts to the repo
• It will push attributes and security objects (not their data)
• It will push cubes, dimensions, hierarchies, views, subsets, processes and chores (unless specified in the [Ignore] section)
• It will ignore processes that start with ‘zTest’ and ‘WIP’ (you can add more wildcards in subsequent lines)
• It will run ‘export_before_push’ before pushing to the repo (thus ensuring source files are up-to-date)
• It will run ‘save_data’ and ‘backup_instance’ processes before pulling (thus modifying) anything in the target tm1 db
• It will run ‘import_after_pull’ immediately after updating the model and source files from the repo
• It will run ‘post_deployment_task’ to complete the deployment with any custom processes needed (i.e.: specific to a release)
• It will update the tm1s.cfg file (for ‘PerformanceMonitorOn’, ‘PerfMonActive’ and ‘TopLog’ fields) depending on the deployment (environment) in which it is run.

Why should you consider it?

There are multiple ways to move code around in TM1, from copying and pasting files to something more advance like Pulse. With the introduction of REST API and TM1Py I’m pretty sure many developers have built their own custom application to spot differences, update objects and/or run processes as part of a deployment. So, why would you care for tm1project? Here are some reasons:

• It is the only method that can functionally work with a git repository. Not only you can use it for version control, you can also deploy from it without needing to restart your instance.
• It provides the ability to natively include non-TM1 objects in your package/deployment. This is quite useful for storing small files (to load a ‘sys’ cube or other environmental data) as well as non-TM1 scripts (like bat, ps1, etc.).
• Future-proof: once IBM includes the ability to execute REST from a TI process, you will be able to promote your code from within a TM1 instance without requiring any third party software or creating custom programs.
• Also, future-proof is the fact that in next gen of TM1 (v12) we are not going to have file access anymore, so switching over to a REST API driven approach will become mandatory.
• Better DevOps integration - since now we can use Git to store and deploy our code we can take advantages of its integration with other common DevOps tool such as Jenkins, GitHub actions, ServiceNow, etc.

Supported tm1project components in TM1Py

• Settings: here you can specify the value any parameters within the tm1s.cfg file.
• Tasks: a process or a chore in the TM1 model.
• Ignore: by default the tm1server will publish all non-control objects (i.e.: anything starting with ‘}’); here you can specify which ones to ignore or which ones to include (by adding the ‘!’ at the beginning)
• Files: non TM1 objects (csv, txt, bat, ps1, etc.) sitting inside the data folder that you want to publish too.
• Deployments: here you can (re) define any component (except for the version) of a tm1project. This is particular useful to distinguish between DEV, QA, UAT and PROD environments.
• Pre & Post Push/Pull: a set of tasks to be executed before and after publishing or deploying an TM1 instance.

Written by Nicolas Bisurgi