Utils to use openMSX in build pipelines.
Go to file
Willem Cazander fbdbdd5b28
All checks were successful
Run test asserts / Test-Asserts (push) Successful in 7s
Fixed wordign.
2024-07-16 16:59:36 +02:00
.forgejo/workflows Converted to documenting variable. 2024-07-15 02:49:21 +02:00
lib Converted to make style like "filter-out" 2024-07-16 16:45:37 +02:00
src Converted to make style like "filter-out" 2024-07-16 16:45:37 +02:00
.gitignore Added project phases and flow generators. 2024-07-07 04:33:03 +02:00
.project Unignored eclipse project file. 2021-07-02 18:58:34 +02:00
Makefile Converted to make style like "filter-out" 2024-07-16 16:45:37 +02:00
README.md Fixed wordign. 2024-07-16 16:59:36 +02:00

MSXBUILD

Utils to use openMSX in build pipelines.

You can use these scripts as-is. the files in the lib are all you need.
But as always, feel free to extend it for your specific needs or contribute a fix or feature.

Features

  • Allows parallel headless openMSX build pipeline
  • openMSX safe exit failure guards
  • Automatic disk image import + export per build step
  • Downloads and caches resources from msxhub.com
  • Auto magic openMSX extensions and machine rom's setup
  • SDCC msx build steps
  • Screenshot and/or video output of full build
  • Embedded documentation of make options

Dependencies

  • make + wget + tar + awk + grep + dos2unix
  • sdcc
  • openmsx
  • ffmpeg
  • rsync

For debian use;

apt-get install make wget tar gawk grep dos2unix sdcc openmsx ffmpeg rsync

Install Classic

TODO

Install Docker

TODO

Usage Setup

Most simple Makefile example;

PATH_MSXBUILD ?= ~/somewhere/local/copy/msxbuild/lib

# Optional set own prefix BEFORE include mxsbuild
.RECIPEPREFIX := _

include $(PATH_MSXBUILD)/msxbuild.mk

# Write normal make rules

Use simple with 0module structure add tree include structure.

PATH_SRC := src
$(call mb_make_call,mb_flow_0module_setup,$(PATH_SRC))

Or more complex do a full flow setup of project with;
(note this will call also the 0module_setup for you)

PATH_BIN := bin
PATH_SRC := src
$(call mb_make_call,mb_setup_default,$(PATH_BIN),$(PATH_SRC))

# Write normal and/or dynamic flow rules in many 0module.mk files.

Rule Writing

TODO: Link to example project on github a few ways to compile and test hello.

Usage Help

When using the mb_setup_default this add a help system which works by running: make @help | grep "*"

 * @help-variable
 * @help-variable-deep
 * @help-variable-rock
 * @help-variable-flow
 * @help-variable-i18n
 * @help-function
 * @help-function-deep
 * @help-function-flow
 * @help-target
 * @help-target-deep
 * @help-target-run
 * @help-target-module
 * @help-target-assert
 * @help-machine
 * @help-all
 * @help-firemake
 * @clean
 * @init
 * @prepare
 * @process
 * @compile
 * @link
 * @build
 * @test
 * @package
 * @package-qa
 * @package-deploy
 * @all
 * @flight-video-build
 * @flight-video-test
 * @flight-video-package-qa
 * @flight-video-all

Running: make @help-all | grep "*" | wc -l to count all documented items of an (example) project;

546

Example Project

When you want to see whats happening do a debug run;

normal run;
	time make
	real    0m8.058s

debug run at 11 Mhz;
	time VERBOSE=on DEBUG=on MB_OPENMSX_HEADLESS=off MB_OPENMSX_THROTTLE=on make
	real    2m34.071s

debug run real msx speed;
	time VERBOSE=on DEBUG=on MB_OPENMSX_HEADLESS=off MB_OPENMSX_THROTTLE=on MB_OPENMSX_SPEED=100 make
	real    2m37.379s

record build run with screenshot on exit;
	time MB_OPENMSX_HEADLESS=off MB_FLIGHT_SCREEN=on make @build
	real    2m40.784s

record matrix test run with video per session and merge to one video file;
	time ASSERT_MSXROM_MATRIX=on make @flight-video-all
	real    0m18.184s

full build parallel;
	time make -j4 @all
	real    0m4.226s

manual test all packages on other MSX1 machine;
	make ASSERT_MSXHUB_MACHINE=Philips_VG_8000 bin/assert/msxhub/@run

view the make graph of an target;
	make -Bnd bin/example/dist-hello-qa-msx1/@assert | make2graph | dot -Tpng -o /tmp/out.png;open /tmp/out.png

Internal conventions make

  • A library MUST use variables for indenting, zero tab space or white space
  • Internal things MUST start with an under score
  • Variables MUST be in upper case
  • Variables MUST use an under score as separator
  • File targets MUST be in a relative binary build folder
  • None file targets MUST start with at-sign with optional build folder prefix
  • None file targets MUST separate words with an hyphen
  • Define eval flow MUST start with two underscores
  • Eval flow MUST be injected by a function
  • Eval flow MUST have an preamble/restore of dynamic white space
  • Functions MUST call function with mb-make-call for DEBUG tracing
  • Functions MUST be in lower case
  • Functions MUST separate words with an hyphen

Errata

  • (?19++) Very sometimes the fail_after "boot_err" watchdog doesn't get canceled
  • Make: @@include-mods will not get chained, it doesn't list ANY dep anymore
  • Make: @@include-tree will not get phony, (and thus @@include-root not on top)
  • boot_exec_setcolor: does not work on MSX1 vdp TMS9918
  • Doesn't work on windows
  • TODO: mbboot80+add arg prefix and move to other repro + msxhub package
  • TODO: Missing C and mixed support
  • TODO: Use TCL for BASIC/ASM/C lcov (Cobertura) code coverage

OpenMSX backlog issues

  • Add setting for "save_imgui_on_exit false" so no imgui.ini file is created
  • Only all all 16 bit unicode fonts because unicode is 21 bit but imgui is not
  • (FIXED:rm-SDL) Bug: screenshots sometimes segfaults when using SDL renderer
  • (FIXED:add-issue) Screenshots don't work without throttle
  • (FIXED++) Audio device init option + warning messages on computer without sound card
  • (#1669) MIDI device init option + warning messages on computer without sound card
  • (#1671) Enhancing the IDE CD-ROM support
  • (#1704) Allow <type>IDEHD</type><DeviceName>OPENMSX IDEHD</DeviceName> now hardcoded in IDEHD.CC:37 + update default for mode 40
  • (#1705) Diskmanipulator export (sync) fully non-cased filename compare (duplicates)
  • (#1705) Diskmanipulator export (sync) don't touch/overwrite unmodified files
  • Diskmanipulator reuse the device defined filename
  • Diskmanipulator Imported folders in openMSX; cd utils;cd ..;dir breaks, use cd utils:cd a:\;dir as workaround
  • run instance-id or run/process-label to replace hardcoded "untitled1" path
  • Remove msg or add ignore sram-fully or blank setting: SRAM file nms8250.cmos not found, assuming blank SRAM content.
  • Video record append mode + optional multiple previds to glue videos as one
  • SDL-NULL as window less renderer so that screenshot and/or video works headless
  • Video record remove blue start frame on start of recording
  • (FIXED++) Allow for set renderer none in TCL or command argument for headless mode
  • (DONE) DocUpdate: Added optional exit code of exit command
  • DocUpdate: Add speed note because it is slow + add example "after quit {export hdd}"
  • DocUpdate: Wrap all TCL commands in catch+stderr+exit1
  • DocUpdate: Add 'headless' example via stdio+new + link to control omsxctl

Make4++ backlog issues

Since .RECIPEPREFIX is added to make, it broke library support and having per call different
dynamic white space rules is also really hard for a human brain.
To fix make5 will use (IBM spec) binary files to express the rules.

  • Add a few new define's to have official make documentation
  • BUG: Unicode read+parse ok but function call fails on include $(MB__BASEPATH)/lib/make/प्रणालीᐥᔆʸᔆᐪᓫᔿ/mb_doc.mk.
  • Add a checked call function call-safe which exits on undefined call argument
  • Add call and call-safe debug trace option
  • Add --warn-shell and --warn-shell-recipe to allow user to force the project to use fast zero shell wrapped commands
  • Add -e option to make internal echo command for escaped sequences
  • Add native OS functions for os-echo,os-file-copy,os-file-delete,os-folder-delete,os-folder-create,etc
  • Add string letters function
  • Add string isascii function (death with unicode)
  • Add lowercase and uppercase functions (full 21bit unicode support)
  • Add XML output for help system for integration with IDE's
  • Add wildcard-treewalker for deeper searches
  • Add namespace support for variables/functions/targets like: moduleX::bin/prog.o: moduleX::src/prog.c
  • Add checksum function to calc checksum for file
  • Add os-copy-diff function to use checksums to copy only changed files
  • Allow make to use checksums not timestamps to detect file updates

Make version 5 will bring make to the last century;

  • Make5: Define (binary) namespaced XML to replace ascii Makefile syntax
  • Make5: Remove guile support fully (maybe replace with embedded MSX BASIC)
  • Make5: Add converter command for upgrade make4 syntax and option to run make4 file.
  • Make5: move all internal's to namespace's
  • Make5: Replace all ultra short 1980's magic templates variables with ultra long human readable name
  • Make5: upgrade to new BDOS split file and folder API in OS kernel VFS layer
  • Make5: Delete all Old-Fashioned Suffix Rules code
  • Make5: Delete .EXPORT_ALL_VARIABLES and export without variable
  • Make5: (Opt?) Delete VPATH search variable (replace with namespace vpath ??)
  • Make5: Delete/refactor all (old) code that make the make code ugly
  • Make5: Adds zillala (emacs like) bug reporting output for when make fails script or internally
  • Make5: Replace all trace/debug/base-print with XML output + optional XSLT + wrap old argu

Make version 6 will bring make to the current century;

  • Make6: Only support 18bit four corner method, fixed i18n words, no letters, no ascii, no unicode
  • Make6: Only support octal based (18 and/or 144 bit) computers (the new MSX'es)
  • Make6: Only support new BDOS version (split folders from files)

Make version 7 will bring make to the next century;

  • Make7: Make without source, all make code is generated from the nether skynet step code definitions
  • Make7: If the OS kernel "ATARI TERMINATOR AI" thinks it is faster to run make rules in VHDL it will be done so