msxbuild/README.md

# MSXBUILD

Utils to use openMSX in build pipelines.

You can use these scripts as-is. the files in the `lib` are all you need.  
But as always, feel free to extend it for your specific needs or contribute a fix or feature.

## Features

* Allows parallel headless openMSX build pipeline
* openMSX safe exit failure guards
* Automatic disk image import + export per build step
* Downloads and caches resources from msxhub.com
* Auto magic openMSX extensions and machine rom's setup
* SDCC msx build steps
* Screenshot and/or video output of full build
* Embedded documentation of make options

## Dependencies

* make + wget + tar + awk + grep + dos2unix
* sdcc
* openmsx
* ffmpeg
* rsync

For debian use;

	apt-get install make wget tar gawk grep dos2unix sdcc openmsx ffmpeg rsync

## Install Classic

TODO

## Install Docker

TODO

## Usage Setup

Most simple `Makefile` example;

	PATH_MSXBUILD ?= ~/somewhere/local/copy/msxbuild/lib
	
	# Optional set own prefix BEFORE include mxsbuild
	.RECIPEPREFIX := _
	
	include $(PATH_MSXBUILD)/msxbuild.mk
	
	# Write normal make rules

Use simple with 0module structure add tree include structure.

	PATH_SRC := src
	$(call mb_make_call,mb_flow_0module_setup,$(PATH_SRC))

Or more complex do a full flow setup of project with;  
(note this will call also the 0module_setup for you)

	PATH_BIN := bin
	PATH_SRC := src
	$(call mb_make_call,mb_setup_default,$(PATH_BIN),$(PATH_SRC))
	
	# Write normal and/or dynamic flow rules in many 0module.mk files.


## Rule Writing

TODO: Link to example project on github a few ways to compile and test hello.

## Usage Help

When using the `mb_setup_default` this add a help system which works by running: `make @help | grep "*"`

	 * @help-variable
	 * @help-variable-deep
	 * @help-variable-rock
	 * @help-variable-flow
	 * @help-variable-i18n
	 * @help-function
	 * @help-function-deep
	 * @help-function-flow
	 * @help-target
	 * @help-target-deep
	 * @help-target-run
	 * @help-all
	 * @help-firemake
	 * @clean
	 * @init
	 * @prepare
	 * @process
	 * @compile
	 * @link
	 * @build
	 * @test
	 * @package
	 * @package-qa
	 * @package-deploy
	 * @all
	 * @flight-video-build
	 * @flight-video-test
	 * @flight-video-package-qa

Running: `make @help-all | grep "*" | wc -l` to count all documented items of an project;

	381

## Example Project

When you want to see whats happening do a debug run;

	normal run;
		time make @package-qa
		real    0m8.058s
	
	debug run at 11 Mhz;
		time VERBOSE=on DEBUG=on MB_OPENMSX_HEADLESS=off MB_OPENMSX_THROTTLE=on make @package-qa
		real    2m34.071s
	
	debug run real msx speed;
		time VERBOSE=on DEBUG=on MB_OPENMSX_HEADLESS=off MB_OPENMSX_THROTTLE=on MB_OPENMSX_SPEED=100 make @package-qa
		real    2m37.379s
	
	record run with screenshot on exit; (black screens without throttle)
		time MB_OPENMSX_HEADLESS=off MB_OPENMSX_THROTTLE=on MB_FLIGHT_SCREEN=on make @package-qa
		real    2m40.784s
	
	record run with video per session and merge to one video file;
		time make @flight-video-package-qa
		real    0m18.184s
	
	build parallel;
		time make -j4 @package-qa
		real    0m4.226s
	
	manual test all packages on other MSX1 machine;
		make DIST_QA_MSXHUB_MACHINE=Philips_VG_8000 bin/dist-qa-msxhub/@run
	
	view the make graph of an target;
		make -Bnd bin/dist-qa-msx1/@assert | make2graph | dot -Tpng -o /tmp/out.png;open /tmp/out.png
	

## Errata

* (?19++) Very sometimes the fail_after "boot_err" watchdog doesn't get canceled
* Make: @@include-mods will not get chained, it doesn't list ANY dep anymore
* Make: @@include-tree will not get phony, (and thus @@include-root not on top)
* boot_exec_setcolor: does not work on MSX1 vdp TMS9918
* Doesn't work on windows
* TODO: mbboot80+add arg prefix and move to other repro + msxhub package
* TODO: Missing C and mixed support
* TODO: Use TCL for BASIC/ASM/C lcov (Cobertura) code coverage


## OpenMSX backlog issues

* Only all all 16 bit unicode fonts because unicode is 21 bit but imgui is not
* (FIXED:rm-SDL) Bug: screenshots sometimes segfaults when using SDL renderer
* Screenshots don't work without throttle
* (FIXED++) Audio device init option + warning messages on computer without sound card
* MIDI device init option + warning messages on computer without sound card
* Enhancing the IDE CD-ROM support
* Allow `<type>IDEHD</type><DeviceName>OPENMSX IDEHD</DeviceName>` now hardcoded in IDEHD.CC:37 + update default for mode 40
* Diskmanipulator export (sync) fully non-cased filename compare (duplicates)
* Diskmanipulator export (sync) don't touch/overwrite unmodified files
* Diskmanipulator reuse the device defined filename
* Diskmanipulator Imported folders in openMSX; `cd utils;cd ..;dir` breaks, use `cd utils:cd a:\;dir` as workaround
* run instance-id or run/process-label to replace hardcoded "untitled1" path
* Remove msg or add ignore sram-fully or blank setting: SRAM file nms8250.cmos not found, assuming blank SRAM content.
* Video record append mode + optional multiple previds to glue videos as one
* SDL-NULL as window less renderer so that screenshot and/or video works headless
* Video record remove blue start frame on start of recording
* (FIXED++) Allow for `set renderer none` in TCL or command argument for headless mode
* (DONE) DocUpdate: Added optional exit code of exit command
* DocUpdate: Add speed note because it is slow + add example "after quit {export hdd}"
* DocUpdate: Wrap all TCL commands in catch+stderr+exit1
* DocUpdate: Add 'headless' example via stdio+new + link to control omsxctl

## Make4++ backlog issues

Since `.RECIPEPREFIX` is added to make, it broke library support and having per call different  
dynamic white space rules is also really hard for a human brain.  
To fix make5 will use (IBM spec) binary files to express the rules.

* Add a few new define's to have official make documentation
* BUG: Unicode read+parse ok but function call fails on `include $(MB__BASEPATH)/lib/make/प्रणालीᐥᔆʸᔆᐪᓫᔿ/mb_doc.mk`.
* Add a checked call function `ccall` which exits on undefined call argument
* Add `call` and `ccall` debug trace option
* Add -e option to make internal echo command for escaped sequences
* Add native OS functions for `os_echo,os_file_copy,os_file_delete,os_folder_delete,os_folder_create,etc`
* Add string letters function
* Add string isascii function (death with unicode)
* Add lowercase and uppercase functions (full 21bit unicode support)
* Add XML output for help system for integration with IDE's
* Add `wildcard_treewalker` for deeper searches
* Add namespace support for variables/functions/targets like: moduleX::bin/prog.o: moduleX::src/prog.c
* Add checksum function to calc checksum for file
* Add `os_copy_diff` function to use checksums to copy only changed files 
* Allow make to use checksums not timestamps to detect file updates
* Define (binary) namespaced XML to replace ascii Makefile syntax
* Make5: Remove guile support fully (maybe replace with embedded MSX BASIC)
* Make5: Add converter command for upgrade make4 syntax and option to run make4 file.
* Make5: move all internal's to namespace's
* Make5: Replace all ultra short 1980's magic templates variables with ultra long human readable name
* Make5: upgrade to new BDOS split file and folder API in OS kernel VFS layer
* Make5: Delete all Old-Fashioned Suffix Rules code
* Make5: Delete .EXPORT_ALL_VARIABLES and `export` without variable
* Make5: (Opt?) Delete VPATH search variable (replace with namespace vpath ??)
* Make5: Delete/refactor all (old) code that make the make code ugly
* Make5: Adds zillala (emacs like) bug reporting output for when make fails script or internally
* Make6: Only support 18bit four corner method, fixed i18n words, no letters, no ascii, no unicode