MSXBUILD

Utils to use openMSX in build pipelines.

You can use these scripts as-is. the files in the lib are all you need.
But as always, feel free to extend it for your specific needs or contribute a fix or feature.

Features

• Allows parallel headless openMSX build pipeline
• openMSX safe exit failure guards
• Automatic disk image import + export per build step
• Downloads and caches resources from msxhub.com
• Auto magic openMSX extensions and machine rom's setup
• SDCC msx build steps
• Screenshot and/or video output of full build
• Embedded documentation of make options

Dependencies

• make + wget + tar + awk + grep + dos2unix
• sdcc
• openmsx
• ffmpeg
• rsync

For debian use;

apt-get install make wget tar gawk grep dos2unix sdcc openmsx ffmpeg rsync

Install Classic

TODO

Install Docker

TODO

Usage Setup

Most simple Makefile example;

PATH_MSXBUILD ?= ~/somewhere/local/copy/msxbuild/lib

# Optional set own prefix BEFORE include mxsbuild
.RECIPEPREFIX := _

include $(PATH_MSXBUILD)/msxbuild.mk

# Write normal make rules

Use simple with 0module structure add tree include structure.

PATH_SRC := src
$(call mb_make_call,mb_flow_0module_setup,$(PATH_SRC))

Or more complex do a full flow setup of project with;
(note this will call also the 0module_setup for you)

PATH_BIN := bin
PATH_SRC := src
$(call mb_make_call,mb_setup_default,$(PATH_BIN),$(PATH_SRC))

# Write normal and/or dynamic flow rules in many 0module.mk files.

Rule Writing

TODO: Link to example project on github a few ways to compile and test hello.

Usage Help

When using the mb_setup_default this add a help system which works by running: make @help | grep "*"

 * @help-variable
 * @help-variable-deep
 * @help-variable-rock
 * @help-variable-flow
 * @help-variable-i18n
 * @help-function
 * @help-function-deep
 * @help-function-flow
 * @help-target
 * @help-target-deep
 * @help-target-run
 * @help-target-module
 * @help-target-assert
 * @help-machine
 * @help-all
 * @help-firemake
 * @clean
 * @init
 * @prepare
 * @process
 * @compile
 * @link
 * @build
 * @test
 * @package
 * @package-qa
 * @package-deploy
 * @all
 * @flight-video-build
 * @flight-video-test
 * @flight-video-package-qa
 * @flight-video-all

Running: make @help-all | grep "*" | wc -l to count all documented items of an (example) project;

546

Example Project

When you want to see whats happening do a debug run;

normal run;
	time make
	real    0m8.058s

debug run at 11 Mhz;
	time VERBOSE=on DEBUG=on MB_OPENMSX_HEADLESS=off MB_OPENMSX_THROTTLE=on make
	real    2m34.071s

debug run real msx speed;
	time VERBOSE=on DEBUG=on MB_OPENMSX_HEADLESS=off MB_OPENMSX_THROTTLE=on MB_OPENMSX_SPEED=100 make
	real    2m37.379s

record build run with screenshot on exit;
	time MB_OPENMSX_HEADLESS=off MB_FLIGHT_SCREEN=on make @build
	real    2m40.784s

record matrix test run with video per session and merge to one video file;
	time ASSERT_MSXROM_MATRIX=on make @flight-video-all
	real    0m18.184s

full build parallel;
	time make -j4 @all
	real    0m4.226s

manual test all packages on other MSX1 machine;
	make ASSERT_MSXHUB_MACHINE=Philips_VG_8000 bin/assert/msxhub/@run

view the make graph of an target;
	make -Bnd bin/example/dist-hello-qa-msx1/@assert | make2graph | dot -Tpng -o /tmp/out.png;open /tmp/out.png

Internal conventions make

• A library MUST use variables for indenting, zero tab space or white space
• Internal things MUST start with an under score
• Variables MUST be in upper case
• Variables MUST use an under score as separator
• File targets MUST be in a relative binary build folder
• None file targets MUST start with at-sign with optional build folder prefix
• None file targets MUST separate words with an hyphen
• Define eval flow MUST start with two underscores
• Eval flow MUST be injected by a function
• Eval flow MUST have an preamble/restore of dynamic white space
• Functions MUST call function with mb-make-call for DEBUG tracing
• Functions MUST be in lower case
• Functions MUST separate words with an hyphen

Errata

• (?19++) Very sometimes the fail_after "boot_err" watchdog doesn't get canceled
• Make: @@include-mods will not get chained, it doesn't list ANY dep anymore
• Make: @@include-tree will not get phony, (and thus @@include-root not on top)
• boot_exec_setcolor: does not work on MSX1 vdp TMS9918
• Doesn't work on windows
• TODO: mbboot80+add arg prefix and move to other repro + msxhub package
• TODO: Missing C and mixed support
• TODO: Use TCL for BASIC/ASM/C lcov (Cobertura) code coverage

OpenMSX backlog issues

• Add setting for "save_imgui_on_exit false" so no imgui.ini file is created
• Only all all 16 bit unicode fonts because unicode is 21 bit but imgui is not
• (FIXED:rm-SDL) Bug: screenshots sometimes segfaults when using SDL renderer
• (FIXED:add-issue) Screenshots don't work without throttle
• (FIXED++) Audio device init option + warning messages on computer without sound card
• (#1669) MIDI device init option + warning messages on computer without sound card
• (#1671) Enhancing the IDE CD-ROM support
• (#1704) Allow <type>IDEHD</type><DeviceName>OPENMSX IDEHD</DeviceName> now hardcoded in IDEHD.CC:37 + update default for mode 40
• (#1705) Diskmanipulator export (sync) fully non-cased filename compare (duplicates)
• (#1705) Diskmanipulator export (sync) don't touch/overwrite unmodified files
• Diskmanipulator reuse the device defined filename
• Diskmanipulator Imported folders in openMSX; cd utils;cd ..;dir breaks, use cd utils:cd a:\;dir as workaround
• run instance-id or run/process-label to replace hardcoded "untitled1" path
• Remove msg or add ignore sram-fully or blank setting: SRAM file nms8250.cmos not found, assuming blank SRAM content.
• Video record append mode + optional multiple previds to glue videos as one
• SDL-NULL as window less renderer so that screenshot and/or video works headless
• Video record remove blue start frame on start of recording
• (FIXED++) Allow for set renderer none in TCL or command argument for headless mode
• (DONE) DocUpdate: Added optional exit code of exit command
• DocUpdate: Add speed note because it is slow + add example "after quit {export hdd}"
• DocUpdate: Wrap all TCL commands in catch+stderr+exit1
• DocUpdate: Add 'headless' example via stdio+new + link to control omsxctl

Make4++ backlog issues

Since .RECIPEPREFIX is added to make, it broke library support and having per call different
dynamic white space rules is also really hard for a human brain.
To fix make5 will use (IBM spec) binary files to express the rules.

• Add a few new define's to have official make documentation
• BUG: Unicode read+parse ok but function call fails on include $(MB__BASEPATH)/lib/make/प्रणालीᐥᔆʸᔆᐪᓫᔿ/mb_doc.mk.
• Add a checked call function call-safe which exits on undefined call argument
• Add call and call-safe debug trace option
• Add --warn-shell and --warn-shell-recipe to allow user to force the project to use fast zero shell wrapped commands
• Add -e option to make internal echo command for escaped sequences
• Add native OS functions for os-echo,os-file-copy,os-file-delete,os-folder-delete,os-folder-create,etc
• Add string letters function
• Add string isascii function (death with unicode)
• Add lowercase and uppercase functions (full 21bit unicode support)
• Add XML output for help system for integration with IDE's
• Add wildcard-treewalker for deeper searches
• Add namespace support for variables/functions/targets like: moduleX::bin/prog.o: moduleX::src/prog.c
• Add checksum function to calc checksum for file
• Add os-copy-diff function to use checksums to copy only changed files
• Allow make to use checksums not timestamps to detect file updates

Make version 5 will bring make to the last century;

• Make5: Define (binary) namespaced XML to replace ascii Makefile syntax
• Make5: Remove guile support fully (maybe replace with embedded MSX BASIC)
• Make5: Add converter command for upgrade make4 syntax and option to run make4 file.
• Make5: move all internal's to namespace's
• Make5: Replace all ultra short 1980's magic templates variables with ultra long human readable name
• Make5: upgrade to new BDOS split file and folder API in OS kernel VFS layer
• Make5: Delete all Old-Fashioned Suffix Rules code
• Make5: Delete .EXPORT_ALL_VARIABLES and export without variable
• Make5: (Opt?) Delete VPATH search variable (replace with namespace vpath ??)
• Make5: Delete/refactor all (old) code that make the make code ugly
• Make5: Adds zillala (emacs like) bug reporting output for when make fails script or internally
• Make5: Replace all trace/debug/base-print with XML output + optional XSLT + wrap old argu

Make version 6 will bring make to the current century;

• Make6: Only support 18bit four corner method, fixed i18n words, no letters, no ascii, no unicode
• Make6: Only support octal based (18 and/or 144 bit) computers (the new MSX'es)
• Make6: Only support new BDOS version (split folders from files)

Make version 7 will bring make to the next century;

• Make7: Make without source, all make code is generated from the nether skynet step code definitions
• Make7: If the OS kernel "ATARI TERMINATOR AI" thinks it is faster to run make rules in VHDL it will be done so