Lua palette quantization

[warmCabin]

Don't mind me, just using your appveyor to avoid setting up Visual Studio on my new PC :)

[warmCabin]

Definitely seeing some improvements by switching away from the green channel rods n' cones biasing thing. When I completely turn off the color caching and run this test script, there is a noticeable performance hit when turboing. I'd say it goes from 25x to 10x speed. As such, I changed the color cache to 5 bits per channel and it seems fine... But I'm still a bit leery, so I might make a Lua API that can turn this on and off just in case 5 bits still makes things weird.

Here are some comparison results. Hopefully it's not too hard to see. With the green bias, the ghosts get kind of yellowy when they stack up.

3-bit green bias (the ways things are in master)

3bit-green_bias.mp4

no cache euclid (best results, noticeable performance dip when turboing)

no_cache-euclid.mp4

5-bit Euclid

5bit-euclid.mp4

3 bit euclid

3bit-euclid.mp4

no cache green bias

no_cache-green_bias.mp4

[warmCabin]

Alright, finally got around to adding some Lua stuff.

gui.clearcolorcache

I'm adding this just in case anyone (namely me) still finds that the colors look a little funny. You can even spam it every frame if you want to. Hopefully we won't need it with the larger color cache.

gui.setcolormatchformula

Something I added for testing purposes. I will almost certainly revert this unless you find it valuable.
The blending modes are:

• Original
• Original, but with delta squared (looks better)
• Basic Euclidian
• "Redmean," an algorithm I stumbled upon in my... OK fine, Gemini recommended it.

Intended to be paired with this new version of my little palette tester script:

colorblending.lua

local formulaNames = {"Luminance squared", "Euclidian", "Redmean"}
formulaNames[0] = "Luminance original"
local formula = 3
local opacity = 0.7
local fullOverride = false

local prevInp = {}

gui.register(function()

    inp = input.get()
    if inp.right and not prevInp.right then
        formula = (formula + 1) % 4
        gui.setcolormatchformula(formula)
        gui.clearcolorcache()
    elseif inp.left and not prevInp.left then
        formula = (formula - 1) % 4
        gui.setcolormatchformula(formula)
        gui.clearcolorcache()
    end
    if inp.up and not prevInp.up then
        opacity = math.min(opacity + 0.1, 1.0)
        gui.clearcolorcache()
    elseif inp.down and not prevInp.down then
        opacity = math.max(opacity - 0.1, 0.0)
        gui.clearcolorcache()
    elseif inp.home and not prevInp.home then
        fullOverride = not fullOverride
        gui.clearcolorcache()
    end
    prevInp = inp

    local centerX, centerY = 64, 100
    local baseX, baseY = centerX + 60 * math.sin(emu.framecount() / 60), centerY + 80 * math.cos(emu.framecount() / 80)
    local size = 8

    gui.opacity(fullOverride and 1.0 or opacity)
    for i = 0, 0x3F do
        local x, y = baseX + (i % 16) * size, baseY + math.floor(i / 16) * size
        gui.box(x, y, x + size + 1, y + size + 1, string.format("P%02X", i), "clear")
    end
    
    gui.opacity(1)
    gui.text(10, 10, formulaNames[formula])
    gui.text(10, 20, string.format("%.1f", opacity))

end)

print "Keyboard Controls:"
print "  left/right: cycle blending mode"
print "  up/down:    change opacity"
print "  home:       toggle full opacity override"

A few screenshots

Original	Euclidian	Redmean

---

And to give you an idea of what the 3-bit to 5-bit cache update is doing: Each grid region is considered the same color. I changed it from the big squares to the small squares. Kinda looks like that Hues & Cues game I keep seeing at Target!

[warmCabin]

Looks like you guys have been busy with the Mac build scripts. I'll go ahead and rebase.

[warmCabin]

@zeromus @bbbradsmith
Can you guys take a look at this?

[zeromus]

why does the cache get wacky? I can't imagine why something like that would happen. if there's an explanation for it I might accept it, but otherwise it seems like a bug-in-waiting.

that would seem to be the only blemish on here (I don't mind keeping the algorithm picker for old-video compatibility or better blazing performance)

[warmCabin]

The cache gets wacky because the colorspace regions are too broad. It stores the first color it happened to encounter in any given region, so over time, you end up with some strange mismatches. By changing it to use the upper 5 bits, the regions get tightened up and it's still just as fast. That's what I was trying to convey with those pretty rainbow squares.

Since the same problem is technically still here (just less obvious), things might still look a little off. That's why I wanted to make gui.clearcolorcache.