Context
The engine gained several user-facing crawl options between 3.49-3 and 3.49-12 that the
GUIs were frozen before. The Windows GUI now exposes them in xroche/httrack-windows#31
(green, reviewed). The Android app should expose the same set so the front-ends stay in
parity. This issue carries everything needed to do it independently.
The engine is vendored here under app/src/main/jni/httrack/, so it already contains these
options once the JNI copy is current — this is purely about surfacing them in the Android
options UI.
Where this lands in this repo
The options UI mirrors the desktop tabs:
app/src/main/java/com/httrack/android/OptionsMapper.java — maps UI fields ⇄ engine options (the seam to extend).
app/src/main/java/com/httrack/android/OptionsActivity.java — options screens.
app/src/main/res/layout/activity_options_proxy.xml, …_spider.xml, …_flowcontrol.xml, …_limits.xml, …_build.xml, …_logindexcache.xml — per-panel layouts.
- UI strings go in
app/src/main/res/values/strings.xml (+ per-locale values-*/).
Options to expose
Emitted to the engine as the CLI flag shown. Short forms verified in the engine's
src/htsalias.c; long forms are the documented public names.
| Feature |
CLI (long / short) |
Kind |
Where (panel) |
Semantics |
| SOCKS5 proxy |
-P socks5://[user:pass@]host[:port] |
proxy string |
proxy |
Scheme prefixed onto the single -P. socks5://≡socks5h:// (remote DNS). Default port 1080 vs 8080. CONNECT-for-HTTPS and SOCKS tunneling are automatic. Reuse the single proxy field; add a protocol selector. |
| Preload cookies |
--cookies-file <path> / -%K |
value |
spider |
Load a Netscape cookies.txt before crawling. A file picker fits Android well. |
| Random pause |
--pause MIN[:MAX] / -%G |
value (seconds) |
flow control |
Random delay between downloads. Distinct from -G (pause after N bytes). |
| Strip query keys |
--strip-query <[host/pattern=]k1,k2,…> / -%g |
value |
limits/expert |
Drop query keys from dedup naming. Simple sid,utm_source; scoped www.example.com/*=sid,ref. |
| Keep www. prefix |
--keep-www-prefix / -%j |
toggle |
links |
Opt out of one part of the -%u URL hack. |
| Keep double slashes |
--keep-double-slashes / -%o |
toggle |
links |
Opt out of -%u. |
| Keep query order |
--keep-query-order / -%y |
toggle |
links |
Opt out of -%u. |
Value options take a separate argument; the three keep-* are bare toggles that only matter
when -%u is on.
Stale default to check
The engine (3.49-11) defaults the User-Agent to
Mozilla/5.0 (compatible; HTTrack/<ver>; +https://www.httrack.com/). Make sure the Android
app doesn't ship the old Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98) default. Also
check the default max transfer rate (engine default 100000 B/s, cap 10000000 B/s).
No UI needed (automatic)
deflate, robots Allow/wildcards, <source>/<track>, full srcset, WHATWG entities,
modern MIME types, HTTPS-over-HTTP-proxy CONNECT, SOCKS tunneling.
Deferred
--why / -%Y (filter-verdict diagnostic, diagnose-and-exit) — leave out or make a "test a
URL" action, matching the Windows decision.
Commit-by-commit, same options, done:
- Modernize the default User-Agent.
- SOCKS5 — proxy-type (HTTP/SOCKS5) selector prepending
socks5:// to -P, reusing the single proxy field.
--cookies-file field.
--pause field (seconds, MIN:MAX).
- Three
--keep-* checkboxes.
--strip-query field.
- Polish: file picker for the cookies path; clearer labels.
OptionsMapper.java is the Android analogue of WinHTTrack's Shell.cpp command-builder —
add the same flag emissions there and the matching widgets in the layout XML + strings.xml.
Context
The engine gained several user-facing crawl options between 3.49-3 and 3.49-12 that the
GUIs were frozen before. The Windows GUI now exposes them in xroche/httrack-windows#31
(green, reviewed). The Android app should expose the same set so the front-ends stay in
parity. This issue carries everything needed to do it independently.
The engine is vendored here under
app/src/main/jni/httrack/, so it already contains theseoptions once the JNI copy is current — this is purely about surfacing them in the Android
options UI.
Where this lands in this repo
The options UI mirrors the desktop tabs:
app/src/main/java/com/httrack/android/OptionsMapper.java— maps UI fields ⇄ engine options (the seam to extend).app/src/main/java/com/httrack/android/OptionsActivity.java— options screens.app/src/main/res/layout/activity_options_proxy.xml,…_spider.xml,…_flowcontrol.xml,…_limits.xml,…_build.xml,…_logindexcache.xml— per-panel layouts.app/src/main/res/values/strings.xml(+ per-localevalues-*/).Options to expose
Emitted to the engine as the CLI flag shown. Short forms verified in the engine's
src/htsalias.c; long forms are the documented public names.-P socks5://[user:pass@]host[:port]-P.socks5://≡socks5h://(remote DNS). Default port 1080 vs 8080. CONNECT-for-HTTPS and SOCKS tunneling are automatic. Reuse the single proxy field; add a protocol selector.--cookies-file <path>/-%K--pause MIN[:MAX]/-%G-G(pause after N bytes).--strip-query <[host/pattern=]k1,k2,…>/-%gsid,utm_source; scopedwww.example.com/*=sid,ref.--keep-www-prefix/-%j-%uURL hack.--keep-double-slashes/-%o-%u.--keep-query-order/-%y-%u.Value options take a separate argument; the three
keep-*are bare toggles that only matterwhen
-%uis on.Stale default to check
The engine (3.49-11) defaults the User-Agent to
Mozilla/5.0 (compatible; HTTrack/<ver>; +https://www.httrack.com/). Make sure the Androidapp doesn't ship the old
Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)default. Alsocheck the default max transfer rate (engine default 100000 B/s, cap 10000000 B/s).
No UI needed (automatic)
deflate, robots
Allow/wildcards,<source>/<track>, fullsrcset, WHATWG entities,modern MIME types, HTTPS-over-HTTP-proxy CONNECT, SOCKS tunneling.
Deferred
--why/-%Y(filter-verdict diagnostic, diagnose-and-exit) — leave out or make a "test aURL" action, matching the Windows decision.
Reference implementation (WinHTTrack, xroche/httrack-windows#31)
Commit-by-commit, same options, done:
socks5://to-P, reusing the single proxy field.--cookies-filefield.--pausefield (seconds,MIN:MAX).--keep-*checkboxes.--strip-queryfield.OptionsMapper.javais the Android analogue of WinHTTrack'sShell.cppcommand-builder —add the same flag emissions there and the matching widgets in the layout XML + strings.xml.