5.0 KiB
Simple Uxn LZ Format
Goals:
- Anyone can implement it
- Small source code size
- Easy to implement from Uxn
- Mildly better than RLE
Non-goals:
- High compression ratio
- High compression speed
Format
It's a stream of commands. The first byte encodes the first command. Read the commands from the input until there's no more input.
There are two commands. Literal and dictionary.
Byte 1 Byte 2+n
┌─────────────────┐ ┌─────
Literal │ 0 x x x x x x x │ │ ....
(Always 1 byte) └─────────────────┘ └─────
Length of literal Bytes to copy to output
(Adjust by adding 1)
Byte 1 Byte 2
Dictionary ┌─────────────────┐ ┌─────────────────┐
(2 bytes version)│ 1 0 x x x x x x │ │ x x x x x x x x │
└─────────────────┘ └─────────────────┘
Length of Offset into
dictionary match dictionary
(Adjust by adding 4) (Adjust by adding 1)
Byte 1 Byte 2 Byte 3
Dictionary ┌─────────────────┬─────────────────┐ ┌─────────────────┐
(3 bytes version)│ 1 1 x x x x x x │ x x x x x x x x │ │ x x x x x x x x │
└─────────────────┴─────────────────┘ └─────────────────┘
Length of dictionary match Offset into
(Adjust by adding 4) dictionary
(Adjust by adding 1)
- The maximum dictionary history size is 256 bytes.
- Dictionary offsets should be treated as the distance from the end of last byte that was output.
- Example: an offset of 0 means go back by 1 bytes into the history.
a b c d e f|g
- Example: an offset of 5 means go back by 6 bytes into the history.
a|b c d e f g
- Example: an offset of 0 means go back by 1 bytes into the history.
22:56 < neauoire> how large do I make the dictionary? 22:57 < cancel> yeah. and the dictionary is just the previous 256 bytes of the file. or, if you haven't progressed through 256 bytes yet, whatever you have 22:57 < cancel> so if you're 20 bytes into the file, your dictionary is the 20 bytes you've already processed 22:57 < cancel> if you're on the first byte of the file, your dictionary size is 0 22:57 < cancel> if you're on byte 500, the dictionary size is 256
22:58 < cancel> if your dictionary size is 0, you're definitely not gonna have a match 22:58 < cancel> if you don't have a match, you need to emit the literal command
22:58 < cancel> if your dictionary size is 0, you're definitely not gonna have a match 22:58 < cancel> if you don't have a match, you need to emit the literal command 22:58 < cancel> and then just slap some bytes down into the output 22:58 < cancel> but... how many? 22:59 < neauoire> it's designed to be stream right? 22:59 < neauoire> mhmm maybe not 22:59 < cancel> yeah, but you have to write the size of the literal first 22:59 < cancel> so... how big should the literal be? 22:59 < cancel> well, you don't know yet 23:00 < cancel> so, just write that the literal is 1 byte long, and then put that first byte of the file you were looking at for a match 23:01 < cancel> now, you're looking at the second byte of the file 23:01 < cancel> repeat the process above 23:01 < cancel> your dictionary is now size 1 23:01 < cancel> and it has that first character in it 23:01 < cancel> let's say your file is 'abcdefg' 23:01 < neauoire> yeah 23:01 < cancel> your dictionary is 'a' 23:01 < cancel> and the next character is 'b' 23:01 < cancel> well, there's no match in the dictionary. 23:02 < cancel> so you need to write a literal again... 23:02 < cancel> but the last thing you wrote was already a literal 23:02 < cancel> so just combine it with the previous literal
23:03 < cancel> ok 23:03 < cancel> you can make a 'compressed' file that doesn't actually compress 23:03 < cancel> it can just be all literals 23:03 < neauoire> it'll take me a while to even just accomplish this bit 23:03 < cancel> it will be bigger than the original input 23:03 < neauoire> ah yes 23:03 < cancel> but it will still be a usable file for the decompressor 23:03 < neauoire> let me try that