I think bit-aligned encodings (i.e. schemes where the output alphabets have a size that is a power of two) are the sweet-spot for encoding schemes due to simplicity of encoding and decoding compared to non-bit-aligned schemes like base84.
Base16 is verbose, but each symbol in the alphabet carries exactly 4 bits (hence why a single byte in hex is two symbols). This is nice for encoding bytes. Base64 requires two non-alphanumeric symbols such as + and / or - and _ because A-Za-z0-9 only provides 62 characters. Each output symbol encodes exactly 6 bits of the original input. But when we're encoding whole bytes we sometimes need to use padding (when the input is not a multiple of 3 bytes and that ambiguity is harmful in the context that decoding occurs in). Then there's Base32, which is what currently seems like a sweet spot to me and is what I'm considering using for identifiers in my own systems. It only requires 32 characters, so can easily take a range of A-Z0-9 or a-z0-9 excluding visually-similar characters. Like Base64 it can also require padding in some settings.
I quite like the general scheme used in the Bech32 and Bech32m framing idea, which allows for a human-readable prefix before a `1` and then the base32 data follows. This can be done because `1` is excluded from the Bech32 alphabet. This prefix can be used to differentiate between kinds of identifier, for example:
But to address the article directly, I'm not sure the complexity is worth the gains in the table in https://github.com/jedisct1/zig-base84#encoding, which states that we save ~6.5% of characters, and for 128 input bytes, the resulting Base64 string is 171 characters, while the Base84 string is 161-166 characters.
I also dislike that the chosen alphabet breaks text selection; GitHub very carefully picked their current token formats so that the whole token is selected with a double-click:
> One other neat thing about _ is it will reliably select the whole token when you double click on it. Other characters we considered are sometimes included in application word separators and thus will stop highlighting at that character. Try out double clicking this-random-text versus this_random_text!
Regardless though we still have the fundamental issue that bytes encoded as Base16, Base32, Base64, or even Base91/Base84 will be _longer_ when encoded. Base32 can encode up to 159 bytes before hitting a filename limit of 255 bytes. I can't remember whether filenames are 16-bit on Windows (UCS-2? UTF-16?), but I speculate on average that maybe you could save a lot of bytes by first ensuring that filenames are UTF-8 before encrypting them, since filenames on a lot of computeres, especially ones running in the west, likely contain a lot of latin characters. You could even switch between encodings to get the best bit-packing in filenames.
I think turbocrypt could switch to using a surrogate file when an encrypted filename exceeds 255 bytes to allow for encrypting any filename permitted by the filesystem, while still preserving the property that the same filename in a different directory has the same encrypted filename. If an encrypted filename is too long to fit on the filesystem, maybe its hash could be stored as the filename instead (which is extremely unlikely to collide). Then we could store the full encrypted filename in a `.name` file in the same directory. We could do similar for directory names that are too long. We could then use a less dense but bit-aligned encoding scheme like Base32 with an all-lowercase alphabet without any punctuation to make filename text selection straightfoward. We would avoid all case sensitivity traps that can occur with things like Base64 and Base84. Let's pick two filename prefixes, say "tcf1" for "turbocrypt filename" and "tch1" for "turbocrypt filename hash". Then we could have a directory layout like this:
tch1q9x8gf2tvdw0 # a file whose encrypted filename exceeds 255 bytes; this filename is a hash of the encrypted filename, e.g. SHA-256.
tch1q9x8gf2tvdw0.name # a file whose contents are the full encrypted filename of tch1q9x8gf2tvdw0.
tcf1tvdhc6mua7l9x8s3qprz # a file whose encrypted filename does not exceed 255 bytes.
Once a system to support long filenames is implemented, the size of the alphabet used for encoding the filenames (like Base84) becomes less important; Base16, Base32, Base64, or another bit-aligned encoding scheme could be used.
As a very small added bonus, you could even implement a bit-aligned codec such as base16 or base64 using SIMD via a bunch of swizzling, shifting, bitmasking, and AND/XORing, making it very fast should the need arise.
Conversely, as a hobbyist photographer, I want to do the exact opposite for most photos I take.
I would like my camera info, especially the body, lens, focal length, and settings in the image. I recently discovered that software like Darktable can even take a gpx file and photo timestamps to add coordinates to photos taken on a camera without a GNSS receiver.
This why I have my phones track themselves (started with Moostrax on the Blackberry then iOS, Moves on iOS until Facebook killed it, now it's OwnTracks on iOS logging to my server + Arc Timeline + Gyroscope + some others, I think) - even without the "where was this photo taken?" helpfulness (for camera shots + phone shots with stripped location), it's also good for "where was that cafe / coffee shop / craft shop / whatever?" kind of questions (obviously assuming you can remember vaguely what date and time...)
I should get better at taking contemporaneous notes, really, but since that hasn't happened in 30+ years, I doubt it's going to stick now.
It's so stupid that we have to do geotagging as a post processing step with separate software. That could be built in to the camera with a GNSS receiver or smartphone Bluetooth link with minimal impact on cost or battery life. And yet even today many high end cameras lack such basic functionality. It's like the camera manufacturers aren't even trying anymore.
A writable file closing itself when it goes out of scope is usually not great, since errors can occur when closing the file, especially when using networked file systems.
You need to close it and check for errors as part of the happy path. But it's great that in the error path (be that using an early return or throwing an exception), you can just forget about the file and you will never leak a file descriptor.
You may need to unlink the file in the error path, but that's best handled in the destructor of a class which encapsulates the whole "write to a temp file, rename into place, unlink on error" flow.
Java solved it by having exceptions be able to attach secondary exceptions, in particular those occurring during stack unwinding (via try-with-resources).
The result is an exception tree that reflects the failures that occurred in the call tree following the first exception.
Or it’s simply an indicator of a schema that has not been excessively normalised (why create an addresses_cities table just to ensure no duplicate cities are ever written to the addresses table?)
DISTINCT, as well as the other aggregation functions, are fantastic for offline analytics queries. I find a lot of use for them in reporting, non-production code.
It depends when you see it, but I agree that DISTINCT shouldn't be used in production. If I'm writing a one off query and DISTINCT gets me over the finish line sparing me a few minutes then that's fine.
There's nothing wrong with using DISTINCT correctly and it does belong in production. The author is complaining about developers that just put in DISTINCT as a matter of course rather than using it appropriately.
One reason to have excessively normalised tables would be to ensure consistency so that you don't have to worry about various records with "London", "LONDON", "lindon" etc.
Because a city/region/state can be uniquely identified with a postal code (hell, in Ireland, the entire address is encapsulated in the postal code), but the reverse is not true.
At scale, repeated low-cardinality columns matter a great deal.
There are ZIP codes that overlap a city and also an unincorporated area. Furthermore, there are zip codes that overlap different states. A data model that renders these unrepresentable may come back to bite you.
This assumption got me in trouble as a junior analyst years ago. I was asked to analyze our customer base and wrote something like the below. Management congratulated me on finding thousands more customers than we'd ever had before.
SELECT
zipcode.rural_urban_code,
COUNT(*) AS n_customer
FROM
customer
INNER JOIN
zipcode USING(zipcode)
GROUP BY 1;
Its less bad that they offered refunds, but why would it that make it ok?
If you buy a car, and the company lights it on fire and then offers you a refund is that ok?
You'll still have the burnt husk if you choose not to take the refund
It's hard to take this comparison seriously because Rocket League is a (mostly) online game for which an active connection to active servers (and thus a cost to the developer). Also, there is no burnt husk.
It's like you paying to get lifetime access to a club, the club closing and reimbursing you.
Since when are companies required to run servers for multiplayer? There is always other ways to play multiplayer. At least there used to be but not any more. It is just a thinly veiled excuse to be able to shut it off.
> It's hard to take this comparison seriously because Rocket League is a (mostly) online game for which an active connection to active servers (and thus a cost to the developer).
This is a different situation, but if this was the stop killing games initiative, the answer would be that when you shut down the game you release the server software.
> Also, there is no burnt husk.
The burnt husk is the program on your computer that opens to the menu and then falls over unable to play. That's what you're left with if you don't take the refund.
> It's like you paying to get lifetime access to a club, the club closing and reimbursing you.
That makes it sound like they merely shut down a rocket league hosting service and someone else could provide the same service. They arranged it so they're the only possible way to play rocket league, even though the game runs on my computer using my resources.
I have tried to keep the Psyonix wikipedia article true to reality if you look at the change history but there are people working for Epic heavily whitewashing it and I didn't want to force (wiki) arbitration or cause a disturbance after the first couple edit/revert battles. The Rocket League one is even harder to keep true.
Basically, they said they were stealing the Mac and Linux Rocket League versions because they wanted to go full directx 10 instead of 9. But the fact that the PS3 is still a first class client running Directx 9 even today shows this is/was a lie. Epic lies quite a bit. In fact when they bought Psyonix they loudly announced there would be no changes, it'd stay rocket league. But of course that lie only lasted 6 months. And now they re-write the wiki pages to pretend it was always the plan.
Anyway, I didn't want a refund. I wanted to keep playing rocket league. And now I cannot play. That's wrong. They bricked my game. And everyone thinks that's A-OK. Just like when they'll brick your modem, or your fridge, or maybe your car. Frankly, having any software in a $thing is a huge risk these days given the status quo.
The Investigatory Powers Act 2016 was one of the big things (before Brexit) that made me realise the UK wasn't a suitable place to run a tech business.
>waiting for 2 different lights just to get to the opposite corner.
A solution sometimes seen in London is a “Pedestrian Scramble”, where pedestrians are explicitly given full (and even diagonal) access to a junction with all other traffic stopped.
In Seattle, they call these "all walks" or officially, "all way walks." I love them, since I don't feel like I have to watch out for drivers making left turns.
With SSO, the party running the SSO decides what the authentication policy is.
For example, where the authentication request is coming from (on-site, managed device), what methods are being used (hardware second factor, Authenticator app).
These are all things that the SSO can check at time of authentication, before a token or session key gets issued to the user. Also, all of these things can be checked again when doing any auth flows for the various linked services.
So with stolen SSO credentials, they might be worth diddly squat to you if you didn’t think to also be on-site or on a managed company device (physically or virtually).
reply