██████╗ ██╗ ██╗███████╗████████╗ ██████╗██╗████████╗██╗ ██╗ ██╔══██╗██║ ██║██╔════╝╚══██╔══╝██╔════╝██║╚══██╔══╝╚██╗ ██╔╝ ██████╔╝██║ ██║███████╗ ██║ ██║ ██║ ██║ ╚████╔╝ ██╔══██╗██║ ██║╚════██║ ██║ ██║ ██║ ██║ ╚██╔╝ ██║ ██║╚██████╔╝███████║ ██║ ╚██████╗██║ ██║ ██║ ╚═╝ ╚═╝ ╚═════╝ ╚══════╝ ╚═╝ ╚═════╝╚═╝ ╚═╝ ╚═╝
Low-level binary I/O library for the RuneScape protocol. Provides fixed-size packet buffers with bit-level operations, RSA encryption, bzip2 compression, CRC-32 checksums, and Jag archive file handling.
- Architecture Overview
- Packet
- Jag Archive Format
- Bzip2 Compression
- CRC-32
- Build System
- File Reference
┌─────────────────────────────────────────────────────────┐
│ rs-io │
│ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ Packet │ │
│ │ │ │
│ │ data: Vec<u8> (fixed-capacity, no realloc) │ │
│ │ pos: usize (byte cursor) │ │
│ │ pos2: usize (bit cursor) │ │
│ │ │ │
│ │ Write: p1 p2 p3 p4 p8 pjstr pdata psmart pbit │ │
│ │ Read: g1 g2 g3 g4 g8 gjstr gdata gsmart gbit │ │
│ │ ALT: p1_alt1/2/3 p2_alt1 ip2_alt1 │ │
│ │ g1_alt1/2/3 g2_alt1 ig2 ig2_alt1 │ │
│ │ RSA: rsaenc rsadec (CRT optimized) │ │
│ │ Bits: bits() bytes() gbit(n) pbit(n,v) │ │
│ └──────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ JagFile │ │ bz2 │ │ crc32 │ │
│ │ │ │ │ │ │ │
│ │ Archive │ │ compress() │ │ getcrc() │ │
│ │ read/write │ │ decompress()│ │ (zlib poly) │ │
│ │ build/save │ │ (FFI to C) │ │ (FFI to C) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │ │ │
│ ┌──────┴────────────────────┘ │
│ │ Bundled C: bzip2 1.0.8 + zcrc32 │
│ │ Compiled at -O3 via cc crate │
│ └────────────────────────────────────│
└─────────────────────────────────────────────────────────┘
The core type for all binary protocol I/O. Fixed-capacity buffer with a position cursor. Uses unchecked indexing for performance — callers must ensure bounds.
// Allocate with exact capacity
let mut buf = Packet::new(256);
// Wrap existing bytes
let mut buf = Packet::from(vec![0x01, 0x02, 0x03]);
// Access
buf.len() // data.len()
buf.pos // current read/write positionpub enum PacketFrame {
Fixed = 0, // no size header
VarByte = 1, // 1-byte size prefix
VarShort = 2, // 2-byte BE size prefix
}All write methods advance pos by the number of bytes written.
| Method | Bytes | Encoding | Description |
|---|---|---|---|
p1(u8) |
1 | Direct | Write unsigned byte |
p1_alt1(u8) |
1 | Negate | Write (-value) & 0xFF |
p1_alt2(u8) |
1 | 128 minus | Write (128 - value) |
p1_alt3(u8) |
1 | Plus 128 | Write (value + 128) |
p2(u16) |
2 | Big-endian | Unsigned 16-bit |
p2_alt1(u16) |
2 | BE, lo+128 | [hi, lo+128] |
ip2(u16) |
2 | Little-endian | Unsigned 16-bit |
ip2_alt1(u16) |
2 | LE, lo+128 | [lo+128, hi] |
p3(i32) |
3 | Big-endian | Lower 24 bits |
p4(i32) |
4 | Big-endian | Signed 32-bit |
ip4(i32) |
4 | Little-endian | Signed 32-bit |
p8(i64) |
8 | Big-endian | Signed 64-bit |
pjstr(&str, u8) |
var | UTF-8 + terminator | Null/10-terminated string |
pdata(&[u8], off, len) |
var | Raw copy | Bulk data write |
All read methods advance pos by the number of bytes read.
| Method | Bytes | Returns | Description |
|---|---|---|---|
g1() |
1 | u8 |
Unsigned byte |
g1s() |
1 | i8 |
Signed byte |
g1_alt1() |
1 | u8 |
Negate: (-val) |
g1_alt2() |
1 | u8 |
128 minus: (128-val) |
g1_alt3() |
1 | u8 |
Minus 128: (val-128) |
g2() |
2 | u16 |
BE unsigned 16-bit |
g2s() |
2 | i16 |
BE signed 16-bit |
g2_alt1() |
2 | u16 |
BE, lo-128 |
ig2() |
2 | u16 |
LE unsigned 16-bit |
ig2s() |
2 | i16 |
LE signed 16-bit |
ig2_alt1() |
2 | u16 |
LE, lo-128 |
g3() |
3 | i32 |
BE 24-bit as i32 |
g4s() |
4 | i32 |
BE signed 32-bit |
ig4s() |
4 | i32 |
LE signed 32-bit |
g8s() |
8 | i64 |
BE signed 64-bit |
gjstr(u8) |
var | String |
Read until terminator |
gdata(&mut [u8], off, len) |
var | — | Bulk data read |
Switch between byte mode and bit mode for sub-byte granularity:
Byte mode: pos tracks byte position
Bit mode: bit_pos tracks bit position (pos * 8)
┌─────────┬─────────┬─────────┬─────────┐
│ byte 0 │ byte 1 │ byte 2 │ byte 3 │
│76543210 │76543210 │76543210 │76543210 │
└─────────┴─────────┴─────────┴─────────┘
▲ ▲
bit_pos=5 bit_pos=13
| Method | Description |
|---|---|
bits() |
Enter bit mode: bit_pos = pos << 3 |
bytes() |
Exit bit mode: pos = (bit_pos + 7) >> 3 |
gbit(n) -> i32 |
Read n bits, handles byte boundaries |
pbit(n, value) |
Write n bits, handles byte boundaries |
Bit operations handle spanning across byte boundaries automatically.
Encryption (rsaenc):
plaintext → BigInt → plaintext^e mod n → [len_byte][ciphertext_bytes]
Decryption (rsadec, CRT optimized):
[len_byte][ciphertext_bytes] → BigInt →
m1 = cipher^dp mod p
m2 = cipher^dq mod q
h = qinv * (m1 - m2) mod p
plaintext = m2 + h * q
→ writes decrypted bytes from pos=0
CRT (Chinese Remainder Theorem) decryption is ~4x faster than naive cipher^d mod n.
Variable-length integer encoding for space efficiency:
psmart / gsmart (unsigned):
┌──────────────────────┬────────────────────────────────┐
│ value 0..127 │ 1 byte: value as u8 │
│ value 128..32767 │ 2 bytes: (value + 32768) as u16│
└──────────────────────┴────────────────────────────────┘
Discriminator: first byte < 128 → 1 byte, >= 128 → 2 bytes
psmarts / gsmarts (signed):
┌──────────────────────┬────────────────────────────────┐
│ value -64..63 │ 1 byte: (value + 64) as u8 │
│ value -16384..16383 │ 2 bytes: (value + 49152) as u16│
└──────────────────────┴────────────────────────────────┘
For variable-length packets, the size is written retroactively:
let start = buf.pos;
buf.pos += 2; // reserve space for size
// ... write payload ...
buf.psize2(start); // writes (pos - start - 2) at start| Method | Header Size | Description |
|---|---|---|
psize1(pos) |
1 byte | Write (self.pos - pos - 1) as u8 |
psize2(pos) |
2 bytes | Write (self.pos - pos - 2) as u16 BE |
psize4(pos) |
4 bytes | Write (self.pos - pos - 4) as i32 BE |
The Jag (Jagex Archive) format stores multiple named files in a single compressed archive.
┌────────────────────────────────────────────────────────────┐
│ Header │
│ unpacked_size: u24 (3 bytes, BE) │
│ packed_size: u24 (3 bytes, BE) │
│ │
│ If packed_size != unpacked_size: │
│ entire data section is bzip2 compressed │
├────────────────────────────────────────────────────────────┤
│ File Table │
│ file_count: u16 (2 bytes, BE) │
│ │
│ For each file: │
│ name_hash: i32 (4 bytes, BE) │
│ unpacked_size: u24 (3 bytes, BE) │
│ packed_size: u24 (3 bytes, BE) │
├────────────────────────────────────────────────────────────┤
│ File Data │
│ For each file: │
│ [packed_size bytes of data] │
│ (individually bzip2'd if not whole-archive compressed) │
└────────────────────────────────────────────────────────────┘
fn hash(name: &str) -> i32 {
let mut hash: i32 = 0;
for c in name.to_uppercase().chars() {
hash = hash.wrapping_mul(61).wrapping_add(c as i32 - 32);
}
hash
}Files are looked up by hash, not by name string.
// Parse from bytes
let jag = JagFile::from(archive_bytes);
// Read a file by name
let packet: Option<Packet> = jag.read("config");
// Read by index
let packet: Option<Packet> = jag.get(0);
// Mutation
jag.write("newfile", & data);
jag.delete("oldfile");
jag.rename("old", "new");
// Build and save
let bytes: Vec<u8> = jag.build();
jag.save("output.jag");When building, the archive tries two compression strategies and picks the smaller result:
Strategy 1: Whole-archive compression
┌─header─┬─file table─┬─file1 raw─┬─file2 raw─┬─...─┐
│ └──────────── bzip2 compressed ────────────┘ │
Strategy 2: Per-file compression
┌─header─┬─file table─┬─file1 bz2─┬─file2 bz2─┬─...─┐
│ │ │ (each file individually) │
Final output = min(strategy1.len(), strategy2.len())
Bundled bzip2 1.0.8 compiled from C source via FFI. Used for Jag archive compression and decompression.
// Decompress
let data = bz2_decompress(
& compressed, // input bytes
unpacked_len, // expected output size
true, // prepend "BZh1" header before decompression
0, // byte offset into input
);
// Compress
let data = bz2_compress(
& raw, // input bytes
true, // strip "BZh1" header from output
);
// Compress with size prefix
let data = bz2_compress_with_size( & raw);
// output: [4-byte BE size][compressed data]The prepend_header / remove_header options handle RuneScape's convention of stripping the bzip2 magic header ("
BZh1") from stored data to save 4 bytes per file.
Compression: raw → RLE → BWT → MTF → Huffman → output
Decompression: input → Huffman → MTF → BWT → RLE → raw
RLE = Run-Length Encoding
BWT = Burrows-Wheeler Transform (block sorting)
MTF = Move-to-Front Transform
let checksum: i32 = getcrc( & data, offset, length);Uses the standard zlib CRC-32 polynomial with a pre-computed 256-entry lookup table. Compiled from bundled C source (
zcrc32.c + crctable.c).
// build.rs
fn main() {
cc::Build::new()
.files([
"csrc/blocksort.c", // BWT block sorting
"csrc/huffman.c", // Huffman coding
"csrc/crctable.c", // CRC-32 lookup table
"csrc/randtable.c", // Random table for bzip2
"csrc/compress.c", // bzip2 compression
"csrc/decompress.c", // bzip2 decompression
"csrc/bzlib.c", // bzip2 high-level API
"csrc/zcrc32.c", // CRC-32 computation
])
.include("csrc")
.opt_level(3)
.define("BZ_NO_STDIO", None)
.warnings(false)
.compile("rsio_native");
}rs-io/
Cargo.toml # deps: rs-crypto, num-bigint; build-dep: cc
build.rs # compiles bzip2 + crc32 from C source
src/
lib.rs # pub mod bz2, crc, jag, packet
packet.rs # Packet struct, all encode/decode methods
jag.rs # JagFile archive handling
bz2.rs # bzip2 compress/decompress FFI wrappers
crc.rs # CRC-32 FFI wrapper
csrc/
bzlib.h # bzip2 public API header
bzlib_private.h # bzip2 internal types and macros
bzlib.c # bzip2 high-level interface
blocksort.c # BWT block sorting algorithm
huffman.c # Huffman tree construction
compress.c # bzip2 compression core
decompress.c # bzip2 decompression core
crctable.c # CRC-32 pre-computed lookup table
randtable.c # Random table for bzip2 internals
zcrc32.c # CRC-32 computation
| Crate | Version | Purpose |
|---|---|---|
rs-crypto |
workspace | RSA key types for rsaenc/rsadec |
num-bigint |
workspace | BigInt arithmetic for RSA operations |
cc |
1.2 | C compiler integration (build-time) |