Skip to content

Commit 1a48796

Browse files
authored
Merge branch 'develop' into sdl_console
2 parents 25b4ddc + f90152c commit 1a48796

251 files changed

Lines changed: 3801 additions & 3122 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/pull_request_template.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
If this PR makes an externally-visible change in behavior or API, please add an appropriate line to `docs/changelog.txt`.

.github/release_template.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,9 +46,9 @@ Text
4646

4747
As always, remember that, just like the vanilla DF game, DFHack tools can also have bugs. It is a good idea to **save often and keep backups** of the forts that you care about.
4848

49-
Many DFHack tools that worked in previous (pre-Steam) versions of DF have not been updated yet and are marked with the "unavailable" tag in their docs. If you try to run them, they will show a warning and exit immediately. You can run the command again to override the warning (though of course the tools may not work). We make no guarantees of reliability for the tools that are marked as "unavailable".
49+
Some DFHack tools that worked in previous (pre-Steam) versions of DF have not been updated yet and are marked with the "unavailable" tag in their docs. If you try to run them, they will show a warning and exit immediately. You can run the command again to override the warning (though of course the tools may not work). We make no guarantees of reliability for the tools that are marked as "unavailable".
5050

51-
The in-game interface for running DFHack commands (`gui/launcher`) will not show "unavailable" tools by default. You can still run them if you know their names, or you can turn on dev mode by hitting Ctrl-D while in `gui/launcher` and they will be added to the autocomplete list. Some tools do not compile yet and are not available at all, even when in dev mode.
51+
The in-game interface for running DFHack commands (`gui/launcher`) will not show "unavailable" tools by default. You can still run them if you know their names, or you can turn on dev mode by hitting Ctrl-D while in `gui/launcher` and they will be added to the autocomplete list. Some tools listed as "unavailable" in the docs do not compile yet and are not accessible at all, even when in dev mode.
5252

5353
If you see a tool complaining about the lack of a cursor, know that it's referring to the **keyboard** cursor (which used to be the only real option in Dwarf Fortress). You can enable the keyboard cursor by entering mining mode or selecting the dump/forbid tool and hitting Alt-K (the DFHack keybinding for `toggle-kbd-cursor`). We're working on making DFHack tools more mouse-aware and accessible so this step isn't necessary in the future.
5454

.github/workflows/build-linux.yml

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,11 @@ on:
1818
default: DFHack/df-structures
1919
structures_ref:
2020
type: string
21+
stonesense_repo:
22+
type: string
23+
default: DFHack/stonesense
24+
stonesense_ref:
25+
type: string
2126
artifact-name:
2227
type: string
2328
append-date-and-hash:
@@ -107,6 +112,13 @@ jobs:
107112
repository: ${{ inputs.structures_repo }}
108113
ref: ${{ inputs.structures_ref }}
109114
path: library/xml
115+
- name: Clone stonesense
116+
if: inputs.stonesense_ref
117+
uses: actions/checkout@v4
118+
with:
119+
repository: ${{ inputs.stonesense_repo }}
120+
ref: ${{ inputs.stonesense_ref }}
121+
path: plugins/stonesense
110122
- name: Get 3rd party SDKs
111123
if: inputs.launchdf
112124
uses: actions/checkout@v4
@@ -135,6 +147,7 @@ jobs:
135147
-G Ninja \
136148
-DCMAKE_INSTALL_PREFIX=build/image \
137149
-DCMAKE_BUILD_TYPE=Release \
150+
-DDFHACK_RUN_URL='https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}' \
138151
${{ inputs.platform-files && '-DCMAKE_C_COMPILER_LAUNCHER=ccache' || '' }} \
139152
${{ inputs.platform-files && '-DCMAKE_CXX_COMPILER_LAUNCHER=ccache' || '' }} \
140153
-DBUILD_LIBRARY:BOOL=${{ inputs.platform-files }} \
@@ -156,7 +169,7 @@ jobs:
156169
- name: Run cpp tests
157170
if: inputs.platform-files
158171
run: ninja -C build test
159-
- name: Finalize cache
172+
- name: Finalize ccache
160173
if: inputs.platform-files
161174
run: |
162175
ccache --show-stats --verbose

.github/workflows/build-windows.yml

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,11 @@ on:
1818
default: DFHack/df-structures
1919
structures_ref:
2020
type: string
21+
stonesense_repo:
22+
type: string
23+
default: DFHack/stonesense
24+
stonesense_ref:
25+
type: string
2126
artifact-name:
2227
type: string
2328
append-date-and-hash:
@@ -88,6 +93,13 @@ jobs:
8893
repository: ${{ inputs.structures_repo }}
8994
ref: ${{ inputs.structures_ref }}
9095
path: library/xml
96+
- name: Clone stonesense
97+
if: inputs.stonesense_ref
98+
uses: actions/checkout@v4
99+
with:
100+
repository: ${{ inputs.stonesense_repo }}
101+
ref: ${{ inputs.stonesense_ref }}
102+
path: plugins/stonesense
91103
- name: Get 3rd party SDKs
92104
if: inputs.launchdf
93105
uses: actions/checkout@v4
@@ -107,17 +119,17 @@ jobs:
107119
win-msvc
108120
- name: Cross-compile
109121
env:
110-
CMAKE_EXTRA_ARGS: -DBUILD_LIBRARY=${{ inputs.platform-files }} -DBUILD_PLUGINS:BOOL=${{ inputs.platform-files && inputs.plugins }} -DBUILD_STONESENSE:BOOL=${{ inputs.stonesense }} -DBUILD_DOCS:BOOL=${{ inputs.docs }} -DBUILD_DOCS_NO_HTML:BOOL=${{ !inputs.html }} -DINSTALL_DATA_FILES:BOOL=${{ inputs.common-files }} -DINSTALL_SCRIPTS:BOOL=${{ inputs.common-files }} -DBUILD_DFLAUNCH:BOOL=${{ inputs.launchdf }} -DBUILD_TESTS:BOOL=${{ inputs.tests }} -DBUILD_XMLDUMP:BOOL=${{ inputs.xml-dump-type-sizes }} ${{ inputs.xml-dump-type-sizes && '-DINSTALL_XMLDUMP:BOOL=1' || '' }}
122+
CMAKE_EXTRA_ARGS: -DBUILD_PDBS:BOOL=${{ inputs.cache-id == 'release' }} -DDFHACK_RUN_URL='https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}' -DBUILD_LIBRARY=${{ inputs.platform-files }} -DBUILD_PLUGINS:BOOL=${{ inputs.platform-files && inputs.plugins }} -DBUILD_STONESENSE:BOOL=${{ inputs.stonesense }} -DBUILD_DOCS:BOOL=${{ inputs.docs }} -DBUILD_DOCS_NO_HTML:BOOL=${{ !inputs.html }} -DINSTALL_DATA_FILES:BOOL=${{ inputs.common-files }} -DINSTALL_SCRIPTS:BOOL=${{ inputs.common-files }} -DBUILD_DFLAUNCH:BOOL=${{ inputs.launchdf }} -DBUILD_TESTS:BOOL=${{ inputs.tests }} -DBUILD_XMLDUMP:BOOL=${{ inputs.xml-dump-type-sizes }} ${{ inputs.xml-dump-type-sizes && '-DINSTALL_XMLDUMP:BOOL=1' || '' }}
111123
run: |
112124
cd build
113125
bash -x build-win64-from-linux.sh
114126
- name: Finalize cache
115127
run: |
116128
cd build
117129
ccache -d win64-cross/ccache --show-stats --verbose
118-
ccache -d win64-cross/ccache --max-size 150M
130+
ccache -d win64-cross/ccache --max-size ${{ inputs.cache-id == 'release' && '500M' || '150M' }}
119131
ccache -d win64-cross/ccache --cleanup
120-
ccache -d win64-cross/ccache --max-size 500M
132+
ccache -d win64-cross/ccache --max-size ${{ inputs.cache-id == 'release' && '2G' || '500M' }}
121133
ccache -d win64-cross/ccache --zero-stats
122134
- name: Save ccache
123135
if: inputs.platform-files && !inputs.cache-readonly
@@ -145,3 +157,9 @@ jobs:
145157
with:
146158
name: ${{ steps.artifactname.outputs.name }}
147159
path: ${{ steps.artifactname.outputs.name }}.tar.bz2
160+
- name: Upload PDBs
161+
if: inputs.artifact-name && inputs.cache-id == 'release'
162+
uses: actions/upload-artifact@v4
163+
with:
164+
name: ${{ steps.artifactname.outputs.name }}_pdb
165+
path: build/win64-cross/pdb

.github/workflows/clean-cache.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ jobs:
1313
- name: Check out code
1414
uses: actions/checkout@v4
1515
- name: Cleanup
16+
env:
17+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
1618
run: |
1719
gh extension install actions/gh-actions-cache
1820
@@ -28,5 +30,3 @@ jobs:
2830
gh actions-cache delete $cacheKey -R $REPO -B $BRANCH --confirm
2931
done
3032
echo "Done"
31-
env:
32-
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/generate-symbols.yml

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ jobs:
5252
structures_ref: ${{ inputs.structures_ref }}
5353
artifact-name: dfhack-symbols-linux64-build
5454
append-date-and-hash: false
55-
cache-id: release
55+
cache-id: test
5656
cache-readonly: true
5757
plugins: false
5858
secrets: inherit
@@ -65,7 +65,7 @@ jobs:
6565
structures_ref: ${{ inputs.structures_ref }}
6666
artifact-name: dfhack-symbols-windows64-build
6767
append-date-and-hash: false
68-
cache-id: release
68+
cache-id: test
6969
cache-readonly: true
7070
plugins: false
7171
secrets: inherit
@@ -163,8 +163,9 @@ jobs:
163163
pip install itch-dl
164164
minor=$(echo "${{ inputs.version }}" | cut -d. -f1)
165165
patch=$(echo "${{ inputs.version }}" | cut -d. -f2)
166-
itch-dl https://kitfoxgames.itch.io/dwarf-fortress --download-to . --api-key $ITCH_API_KEY
167-
tar xjf "kitfoxgames/dwarf-fortress/files/dwarf_fortress_${minor}_${patch}_linux.tar.bz2" -C DF_itch
166+
fname="dwarf_fortress_${minor}_${patch}_linux.tar.bz2"
167+
itch-dl https://kitfoxgames.itch.io/dwarf-fortress --download-to . --api-key $ITCH_API_KEY --filter-files-glob "${fname}"
168+
tar xjf "kitfoxgames/dwarf-fortress/files/${fname}" -C DF_itch
168169
tar xjf dfhack-symbols-linux64-build.tar.bz2 -C DF_itch
169170
xml/symbols_gen_linux.sh ${{ inputs.version }} ITCH DF_itch
170171
@@ -270,8 +271,9 @@ jobs:
270271
pip install itch-dl
271272
minor=$(echo "${{ inputs.version }}" | cut -d. -f1)
272273
patch=$(echo "${{ inputs.version }}" | cut -d. -f2)
273-
itch-dl https://kitfoxgames.itch.io/dwarf-fortress --download-to . --api-key $ITCH_API_KEY
274-
unzip -d DF_itch "kitfoxgames/dwarf-fortress/files/dwarf_fortress_${minor}_${patch}_windows.zip"
274+
fname="dwarf_fortress_${minor}_${patch}_windows.zip"
275+
itch-dl https://kitfoxgames.itch.io/dwarf-fortress --download-to . --api-key $ITCH_API_KEY --filter-files-glob "${fname}"
276+
unzip -d DF_itch "kitfoxgames/dwarf-fortress/files/${fname}"
275277
xml/symbols_gen_windows.sh ${{ inputs.version }} ITCH DF_itch
276278
277279
# Classic

.github/workflows/steam-deploy.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,12 @@ jobs:
7676
done
7777
- name: Get short SHA of commit
7878
run: echo "SHORT_SHA=`echo ${{ github.sha }} | cut -c1-8`" >>$GITHUB_ENV
79+
- name: steamcmd cache
80+
uses: actions/cache@v4
81+
with:
82+
path: /home/runner/work/_temp/_github_home
83+
key: steamcmd-${{ github.sha }}
84+
restore-keys: steamcmd
7985
- name: Steam deploy
8086
uses: game-ci/steam-deploy@v3
8187
with:

.github/workflows/test-suite.yml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,11 +72,13 @@ jobs:
7272
if: inputs.os == 'ubuntu'
7373
run: Xvfb :0 -screen 0 1600x1200x24 &
7474
- name: Run lua tests
75-
timeout-minutes: 10
75+
uses: nick-fields/retry@v3
7676
env:
7777
DISPLAY: :0
7878
TERM: xterm-256color
79-
run: python ci/run-tests.py --keep-status "${{ env.DF_FOLDER }}"
79+
with:
80+
timeout_minutes: 1
81+
command: python ci/run-tests.py --keep-status "${{ env.DF_FOLDER }}"
8082
- name: Check RPC interface
8183
run: python ci/check-rpc.py "${{ env.DF_FOLDER }}/dfhack-rpc.txt"
8284
- name: Upload test artifacts
Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
name: Watch DF Releases
2+
3+
on:
4+
schedule:
5+
- cron: '8/10 * * * *'
6+
workflow_dispatch:
7+
8+
jobs:
9+
check-steam:
10+
if: github.repository == 'DFHack/dfhack'
11+
name: Check Steam (${{ matrix.df_steam_branch }}) for new DF releases
12+
runs-on: ubuntu-latest
13+
concurrency: watch-release-steam-${{ matrix.df_steam_branch }}
14+
strategy:
15+
fail-fast: false
16+
matrix:
17+
# df_steam_branch: which DF Steam branch to watch
18+
# platform: leave blank to default to all
19+
# structures_ref: leave blank to default to master
20+
# dfhack_ref: leave blank if no structures update is desired
21+
# steam_branch: leave blank if no DFHack steam push is desired
22+
include:
23+
- df_steam_branch: public
24+
- df_steam_branch: beta
25+
steps:
26+
- name: Fetch state
27+
uses: actions/cache/restore@v4
28+
with:
29+
path: state
30+
key: watch-release-steam-${{ matrix.df_steam_branch }}
31+
- name: Compare branch metadata
32+
uses: nick-fields/retry@v3
33+
with:
34+
timeout_minutes: 2
35+
command: |
36+
blob=$(wget 'https://api.steamcmd.net/v1/info/975370?pretty=1' -O- | \
37+
awk '/^ *"branches"/,0' | \
38+
awk '/^ *"${{ matrix.df_steam_branch }}"/,0')
39+
buildid=$(echo "$blob" | \
40+
fgrep buildid | \
41+
head -n1 | \
42+
cut -d'"' -f4)
43+
timestamp=$(echo "$blob" | \
44+
fgrep timeupdated | \
45+
head -n1 | \
46+
cut -d'"' -f4)
47+
test -z "$buildid" && echo "no buildid result" && exit 1
48+
test -z "$timestamp" && echo "no timestamp result" && exit 1
49+
test "$buildid" -gt 0 || exit 1
50+
test "$timestamp" -gt 0 || exit 1
51+
echo "buildid and timestamp of last branch update: $buildid, $timestamp"
52+
mkdir -p state
53+
touch state/buildid state/timestamp
54+
last_buildid=$(cat state/buildid)
55+
last_timestamp=$(cat state/timestamp)
56+
if [ -z "$last_timestamp" ]; then
57+
echo "no stored timestamp"
58+
last_buildid=0
59+
last_timestamp=0
60+
else
61+
echo "stored buildid and timestamp of last branch update: $last_buildid, $last_timestamp"
62+
fi
63+
if [ "$buildid" -ne "$last_buildid" -a "$timestamp" -gt "$last_timestamp" ]; then
64+
echo "branch updated"
65+
echo "$buildid" >state/buildid
66+
echo "$timestamp" >state/timestamp
67+
echo BUILDID=$buildid >> $GITHUB_ENV
68+
fi
69+
- name: Discord Webhook Action
70+
uses: tsickert/discord-webhook@v5.3.0
71+
if: env.BUILDID
72+
with:
73+
webhook-url: ${{ secrets.DISCORD_TEAM_PRIVATE_WEBHOOK_URL }}
74+
content: "<@&${{ secrets.DISCORD_TEAM_ROLE_ID }}> Steam ${{ matrix.df_steam_branch }} branch updated (build id: ${{ env.BUILDID }})"
75+
- name: Launch symbol generation workflow
76+
if: env.BUILDID && matrix.dfhack_ref
77+
env:
78+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
79+
run: |
80+
gh workflow run generate-symbols.yml \
81+
-R DFHack/dfhack \
82+
-r ${{ matrix.dfhack_ref }} \
83+
-f structures_ref=${{ matrix.structures_ref }} \
84+
-f version=auto \
85+
-f platform=${{ matrix.platform }} \
86+
-f channel=steam \
87+
-f df_steam_branch=${{ matrix.df_steam_branch }} \
88+
-f steam_branch=${{ matrix.steam_branch }}
89+
- name: Save state
90+
uses: actions/cache/save@v4
91+
if: env.BUILDID
92+
with:
93+
path: state
94+
key: watch-release-steam-${{ matrix.df_steam_branch }}-${{ env.BUILDID }}
95+
96+
check-non-steam:
97+
if: github.repository == 'DFHack/dfhack'
98+
name: Check ${{ matrix.channel }} for new DF releases
99+
runs-on: ubuntu-latest
100+
concurrency: watch-release-${{ matrix.channel }}
101+
strategy:
102+
fail-fast: false
103+
matrix:
104+
include:
105+
- channel: itch
106+
url: 'https://kitfoxgames.itch.io/dwarf-fortress'
107+
prefix: 'dwarf_fortress'
108+
- channel: classic
109+
url: 'https://www.bay12games.com/dwarves/'
110+
prefix: 'df'
111+
steps:
112+
- name: Fetch state
113+
uses: actions/cache/restore@v4
114+
with:
115+
path: state
116+
key: watch-release-${{ matrix.channel }}
117+
- name: Compare versions
118+
uses: nick-fields/retry@v3
119+
with:
120+
timeout_minutes: 2
121+
command: |
122+
version=$(wget "${{ matrix.url }}" -qO- | tr '"' '\n' | fgrep 'tar.bz2' | head -n1 | sed -r 's/${{ matrix.prefix }}_([0-9]{2})_([0-9]{2})_linux.tar.bz2/\1.\2/')
123+
echo "latest ${{ matrix.channel }} version: $version"
124+
if ! grep -qE '^[0-9]+\.[0-9]+$' <<<"$version"; then
125+
echo "invalid version"
126+
exit 1
127+
fi
128+
mkdir -p state
129+
touch state/last_version
130+
last_version=$(cat state/last_version)
131+
if [ -z "$last_version" ]; then
132+
echo "no stored version"
133+
last_version=0
134+
else
135+
echo "stored ${{ matrix.channel }} version: $last_version"
136+
fi
137+
if [ "$(tr -d '.' <<<"$version")" -gt "$(tr -d '.' <<<"$last_version")" ]; then
138+
echo "${{ matrix.channel }} has been updated"
139+
echo "$version" >state/last_version
140+
echo NEW_VERSION=$version >> $GITHUB_ENV
141+
fi
142+
- name: Discord Webhook Action
143+
uses: tsickert/discord-webhook@v5.3.0
144+
if: env.NEW_VERSION
145+
with:
146+
webhook-url: ${{ secrets.DISCORD_TEAM_PRIVATE_WEBHOOK_URL }}
147+
content: "<@&${{ secrets.DISCORD_TEAM_ROLE_ID }}> ${{ matrix.channel }} updated to ${{ env.NEW_VERSION }}"
148+
- name: Save state
149+
uses: actions/cache/save@v4
150+
if: env.NEW_VERSION
151+
with:
152+
path: state
153+
key: watch-release-${{ matrix.channel }}-${{ env.NEW_VERSION }}

0 commit comments

Comments
 (0)