Custom CI/CD for Unity

Now I will tell you, how is development pipeline looks like in a small gamedev company and how we improve and automate it. Yeah, now is 2020, so stop to use mouse…

About the mouse, this is not such a joke. Until recently, the process of testing and developing games in our company looked something like this:

Android

1. Developers write a code
2. Make build for Android
3. Upload to shared folder on Google Drive
4. Tester download last build
5. Test it and add tasks to Redmine
6. Goto: step 1

With iOS is more complicated:

iOS

1. Developers write a code
2. Make Xcode project
3. Postbuild script adds to it localizations, SDK, etc
4. Compile Xcode project
5. Archive
6. Export to IPA file (4–6 steps is a little bit slowly, haha)
7. Upload IPA to shared folder on Google Drive
8. Or upload to TestFlight, here you can wait a little time too
9. Tester download last build
10. Install IPA on device
11. Test it and add tasks to Redmine
12. Goto: step 1

Such a sadness. And if for Android, the process is still somehow acceptable (no), then for iOS — everything is very slow, inconvenient and wrong. In addition, most of the operations are done manually by developer (here about the mouse) and time that may be spent on development takes to wait for compilation, export of Xcode project, and so on. Here I have to say, that we are all placed in same office i.e. you can come up to ask /say something before starting this whole hellish process.

But, in general, it somehow works and releases games :)

And here recently, with the filing of Leopotam, another lover of construction and optimization of everything useless, I decided to automate this process. The result is a BASH script that can:

• Pull changes form GIT repo
• Run tests
• Make APK for Android, in develop and release modes
• Make iOS Xcode project, in develop and release modes
• Compile it
• Archive it and export to IPA
• Generate manifest for it
• Generate HTML pages with links for download and install
• Upload whole that data to your server via sshpass
• Save logs to temp folder
• Send notify by Telegram bot

And all of this by one command. Without starting Unity and Xcode. Somehow more fun, huh?

Now, when build is ready, Telegram bot send notify to testers with links for Android and iOS builds. Everything is needed is click on it and install the application directly from the device. It also works under iOS, the special manifest is generated for this.

Actually, this is my first experience in BASH programming, so I don’t know how it is, in Feng Shui or not, but it works properly. Anyway, the script could be updated and modified to meet the needs of your projects. Below are a few key points on what is being done:

How to run build process for target platform

In docs you can find all the argument to use for build. Run build Android APK, e.g.:

$UNITY -batchmode -quit -projectPath "$PATH" -executeMethod Game.BuildActions.AndroidDevelopment -buildTarget android -logFile "$LOGS_PATH/android_development.log"

Here -buildTarget set target platform for build, and -executeMethod call static method in Unity project, which run build pipeline with your special params:

static void AndroidDevelopment () {  PlayerSettings.SetScriptingBackend (BuildTargetGroup.Android, ScriptingImplementation.IL2CPP);  PlayerSettings.SetScriptingDefineSymbolsForGroup (BuildTargetGroup.Android, "DEV");  EditorUserBuildSettings.SwitchActiveBuildTarget (BuildTargetGroup.Android, BuildTarget.Android);  EditorUserBuildSettings.development = true;  EditorUserBuildSettings.androidETC2Fallback = AndroidETC2Fallback.Quality32Bit;  BuildReport report = BuildPipeline.BuildPlayer (GetScenes (), ANDROID_DEVELOPMENT_FILE, BuildTarget.Android, BuildOptions.None);  int code = (report.summary.result == BuildResult.Succeeded) ? 0 : 1;  EditorApplication.Exit (code);    
}

Here you can set any params and options for build. The result of build returns to BASH script.

How to compile and export Xcode project

After Unity has successfully completed Xcode project, it must be compiled, archived and exported to IPA file:

xcodebuild -project "$IOS_PATH/Unity-iPhone.xcodeproj" -quiet > "$LOGS_PATH/ios_build_release.log" 2>&1
xcodebuild -project "$IOS_PATH/Unity-iPhone.xcodeproj" -scheme "Unity-iPhone" archive -archivePath "$IOS_RELEASE/Unity-iPhone.xcarchive" -quiet > "$LOGS_PATH/ios_archive_release.log" 2>&1
xcodebuild -exportArchive -archivePath "$IOS_RELEASE/Unity-iPhone.xcarchive" -exportOptionsPlist "$IOS_RELEASE/options.plist" -exportPath $IOS_RELEASE -allowProvisioningUpdates -quiet > "$LOGS_PATH/ios_export_release.log" 2>&1

Here options.plist —is a special manifest file where you can set export method, TeamID, etc. All available params and arguments you can find by run command:

xcodebuild -help

In all these steps you use default Unity project file: Unity-iPhone.xcodeproj. If you use xcworkspace, so you should check it for exists and process.

How to send Telegram notification via BASH script

You have to find BotFather bot, enter /start, or /newbot, fill all required fields, get token and some instructions. To send message via bot, you should execute command:

curl $BOT_PROXY https://api.telegram.org/bot$BOT_TOKEN/sendMessage -m 60 -s -X POST -d chat_id=$CHAT_ID -d text="$1" > "$LOGS_PATH/bot.log" 2>&1

Here CHAT_ID is chat id (yeah), where you will send messages. You can add your new bot to group with testers and give permission to read messages for it. After that, go to:

https://api.telegram.org/bot[BOT_TOKEN]/getUpdates

And get group ID and members IDs. After, you can set ID of group or some member to send messages. Is not a problem how to setup proxy for bot, you can google it.

Other methods in script patches templates, generate HTML, and upload files.

Below are some params what you can quickly change:

#PARAMS TO CHANGE 
BRANCH='master'  
COMPANY='my_company' 
GAME_NAME='new_game' 
BUNDLE='com.mygames.game' 
TEAM='ios_team_id' 
REMOTE_PATH='url_my_builds_server'  
SSH_LOGIN='my_login' 
SSH_PASS='my_pass' 
SSH_HOST='my_builds_server.ru' SSH_PATH='~/domains/my_builds_server.ru/builds'  TEMPLATE_FILE=$(PWD)'/template.html' MANIFEST_FILE=$(PWD)'/manifest.plist' VERSION_FILE=$(PWD)'/version.txt'  
LOGS_PATH=$PROJECT_PATH'/Logs' ANDROID_PATH=$PROJECT_PATH'/Builds/Android' BUILDS_PATH=$PROJECT_PATH'/Builds' IOS_PATH=$PROJECT_PATH'/Builds/iOS' IOS_BUILD_PATH=$PROJECT_PATH'/Builds/iOS/build' IOS_DEVELOPMENT=$PROJECT_PATH'/Builds/iOS/build/development' IOS_RELEASE=$PROJECT_PATH'/Builds/iOS/build/release'  BOT_TOKEN='my_bot_token' BOT_PROXY='--proxy 185.189.211.70:8080' CHAT_ID='123456798'  UNITY='/Applications/Unity/Hub/Editor/2019.3.0f1/Unity.app/Contents/MacOS/Unity'

Demo and sources

Yeah, everything of that, you can make with Gitlab CI/CD, there are templates, scripts, runners, for it. But on Gitlab you should migrate too… Maybe we’ll come to this, while it’s so much better than it was. Moreover, the entire script was written quickly enough.

This article is translation from my blog: https://mopsicus.ru. I will try to translate and write here more often if you like post as this.