Gamedata Build Pipeline
This page is outdated. Please refer to Kaetemi's Manuals
*** IMPORTANT ***
*** IMPORTANT ***
The information below is outdated and for developer reference only.
*** IMPORTANT ***
The gamedata build pipeline consists of a set of scripts, with the ultimate purpose of automatically bringing together your assets and leveldesign data into packaged client datafiles, as well as building all the data as required by the game services.
Python 2.7 has been confirmed to work ([http://www.python.org/download/releases/2.7/]).
You will need the following:
- 3DS Max (For converting 3d assets)
- the Ryzom_assets archive
- Built tools from BuildForWindowsCmake
Projects
The asset pipeline is divided into three major project categories, each containing one or more independent projects that depend on the previous category. The possibility exists to build multiple projects per category in parallel on multiple physical machines. A project has it's own configuration set, allowing multiple independent landscape lighting models or character armor color texture generation.
- common
- characters
- characters_maps_hr
- characters_maps_lr
- construction
- fauna
- fonts
- interface
- objects
- outgame
- sfx
- sky
- ecosystems
- jungle
- desert
- lacustre
- primes_racines
- continents
- newbieland
Processes
Each project may run one or more sequential build processes. A process defines the data which needs to be built, and which tools need to be run for the process to run successfully.
The following processes are currently part of the gamedata build pipeline:
map
: Builds texture color variations for textures inPanoplyFileList
according topanoply.cfg
, and compresses the output to.dds
file format. Compresses.tga
files inMapSourceDirectories
to.dds
files. Installs.png
and.dds
files fromMapSourceDirectories
andMapUncompressedSourceDirectories
,.dds
files fromMapBuildDirectory
andMapPanoplyBuildDirectory
, and.png
,.dds
and.tga
files fromMapUncompressedSourceDirectories
toMapClientDirectory
.- ...
- ...
- ...
- ...
- ...
- ...
Commands
There are four basic commands which run a specific portion of the entire gamedata build pipeline.
- Setup (
0_setup.py
): This script allows you to configure the directories used by the automated pipeline, and creates all the subdirectories required by the available projects and their processes. - Export (
1_export.py
): The main task of this portion is to bring all the assets into a format that can be read by NeL. For instance, it exports all the .shape, .zone, .anim, .skel, .veget, etcetera from the 3ds Max files, as well as preprocessing some data for the build script of the processes. - Build (
2_build.py
): Processes all the content for ingame usage. Welds and lights landscape zones, generates new sheet ids, and compresses large file formats down to a more manageable filesize. It builds packed sheets, compresses .tga files specified by specific processes down to .dds file format, and sample banks for sound effects. - Install (
3_install.py
): Grabs together all the files required by the game client, and game services, ready into an installable file structure.
sfb notes
To prep the MAX files for 2010:
12:44 < Kaetemi> nah, that's for opening one single file 12:44 < Kaetemi> you just need to run max 12:44 < Kaetemi> go to the open maxscript 12:44 < Kaetemi> then choose for batched 12:44 < Kaetemi> enter W:\database 12:44 < Kaetemi> check subdirs 12:44 < Kaetemi> uncheck test 12:44 < Kaetemi> uncheck stop on error 12:44 < Kaetemi> and uncheck something else 12:44 < Kaetemi> and run 12:44 < sfb> hahahaha 12:44 < Kaetemi> and wait for a day or so
Exporting Assets Using The Build Pipeline Step By Step
NOTE: If you have not done so before you will need to follow Convertingryzom_assetsDatabase
(To use the defaults you can use this .bat script, you will need to run this every time you wish to use the drive path's or you can put it in your startup folder as ryzom.bat)
subst @echo off subst R: C:\ryzom subst L: C:\ryzom\code\ryzom\common\data_leveldesign subst W: C:\ryzom\ryzom_assets
1. Create a folder in drive C:\ named "export" (C:\export)
2. Navigate to C:\ryzom\code\nel\tools\build_gamedata and double click on "0_setup.py".
The console will ask for directory's, Its pretty self explanatory, just fill it out and it will set everything up for you.
Note> Everything in the <> are the suggested defaults.
- Database Directory (W:/database): is the assets directory
- Primary Tool Directory: is the location that your compiled binaries from msvc are located at
Build Quality (1): '1' Primary Tool Directory (c:/RyzomCore/build_gamedata/bin): 'c:/RyzomCore/build_gamedata/bin' Secondary Tool Directory (c:/RyzomCore/common/exedll): 'c:/RyzomCore/common/exedll' Tool Suffix (.exe): '.exe' Script Directory (c:/RyzomCore/build_gamedata): 'c:/RyzomCore/build_gamedata' Workspace Directory (c:/RyzomCore/build_gamedata/workspace): 'c:/RyzomCore/build_gamedata/workspace' Database Directory (W:/database): 'W:/database' Export Build Directory (W:/export): 'W:/export' Install Directory (W:/install): 'W:/install' Data Shard Directory (W:/export_data_shard): 'W:/export_data_shard' Client Dev Directory (W:/client_dev): 'W:/client_dev' Client Patch Directory (W:/client_patch): 'W:/client_patch' Client Install Directory (W:/client_install): 'W:/client_install' Leveldesign Directory (L:/leveldesign): 'L:/leveldesign' Leveldesign DFN Directory (L:/leveldesign/dfn): 'L:/leveldesign/dfn' Leveldesign World Directory (L:/leveldesign/world): 'L:/leveldesign/world' Primitives Directory (L:/primitives): 'L:/primitives' Gamedev Directory (c:/RyzomCore/common/data_client/gamedev): 'c:/RyzomCore/common/data_client/gamedev' Data Common Directory (c:/RyzomCore/common/data_common): 'c:/RyzomCore/common/data_common' Primary Windows exe/dll/cfg Directory (C:/Program Files (x86)/Microsoft Visual Studio 9.0/VC/redist/x86): 'C:/Program Files (x86)/Microsoft Visual Studio 9.0/VC/redist/x86' Secondary Windows exe/dll/cfg Directory (c:/RyzomCore/common/exedll): 'c:/RyzomCore/common/exedll' Tertiary Windows exe/dll/cfg Directory (D:/source/external_shared/bin/x86): 'D:/source/external_shared/bin/x86' Quaternary Windows exe/dll/cfg Directory (R:/code/nel/lib): 'R:/code/nel/lib' Quinary Windows exe/dll/cfg Directory (R:/code/ryzom/bin): 'R:/code/ryzom/bin' Senary Windows exe/dll/cfg Directory (R:/code/ryzom/client): 'R:/code/ryzom/client' Septenary Windows exe/dll/cfg Directory (R:/code/ryzom/tools/client/client_config/bin): 'R:/code/ryzom/tools/client/client_config/bin' 3dsMax Available (1): '1' 3dsMax Directory (C:/Program Files (x86)/Autodesk/3ds Max 2010): 'C:/Program Files (x86)/Autodesk/3ds Max 2010' 3dsMax User Directory (C:/Users/Matt/AppData/Local/Autodesk/3dsMax/2010 - 32bit/enu): 'C:/Users/Matt/AppData/Local/Autodesk/3dsMax/2010 - 32bit/enu' 3dsMax Executable (3dsmax.exe): '3dsmax.exe'
3. Navigate to C:\ryzom\code\nel\tools\build_gamedata and double click on "1_export.py".
This exports all the data from the .max files in "Database Directory".
4. Navigate to C:\ryzom\code\nel\tools\build_gamedata and double click on "2_build.py".
This runs all the compile operations like converting images to direct draw format, lighting and sound effects.
5. Navigate to C:\ryzom\code\nel\tools\build_gamedata and double click on "3_install.py".
This installs all the generated data and folder structures into "Client Install Directory" for testing in game.
Please use the new assets archive, the latest plugins for max, the latest tools, and the latest data repository.
https://ryzomcore.atlassian.net/wiki/display/RC/Downloads+for+Windows
http://dl.kaetemi.be/ryzom/assets/ryzom_assets_rev3.7z
https://bitbucket.org/ryzom/ryzomcore-data