If it's on HN and is a meme at this point, it will end up in the training set.

It's kind of fun to imagine that there is an intern in every AI company furiously trying to get nice looking svg pelicans on bicycles.

OK that is recognizably a pelican, pretty great!

This feels like the best pelicanbike yet. The singularity might be closer than we imagine.

Time for a leaderboard?

Ask and you'll receive: https://pelicans.borg.games/

It would be interesting to have two generations per model without cherry picking, so that the Elo estimation can include an easy-to-compute standard deviation estimation.

Nice! Is there a way I can click on the leaderboard items so I can view them?

Added

Lol, can you add a "both of these are terrible" option?

That's what "Reload Pelicans" is for!

I think they (LLMs providers) are manually tuning these cases/examples.

Pelinkan on a bike - > some dude (from these labs) creates it, and it becomes part of the training data.

Meme benchmarks like this and Strawberry are funny but very easy to game, I bet they're all over training sets nowadays.

If you train a model on the SVGs of pelicans on a bicycle that are out there already you're going to get a VERY weird looking pelican on a bicycle: https://simonwillison.net/tags/pelican-riding-a-bicycle/

Thanks for the mention on your blog. You’re the original GPT pelican artist

Truly worth the price. We live in the future.

Honestly the first one where I would have guessed "this is a pelican riding a bicycle" if presented with just the image and 0 other context. This and the voxel tower are fairly impressive - we're seeing some semblance of visual / spatial understanding with this model.

Interestingly it seems to draw the bike's seat too (around line 34) which then gets covered by the pelican.

Easily the best one yet!

First I've seen of human quality. Maybe we are reaching API. (Artificial Pelican Intelligence)

This is not better. It doesn't even get the chassis of the bike right, let alone forgetting the pelican should probably be attached to it.

Uh that doesn't look better. it has more texture but the composition is bad/incomplete

Can it do circuit diagrams? Because that's one practical area where I think the AI models are lacking.

Not yet, or schemas. It can do netlists, though! But it's much harder to go from "Netlist -> Diagram/Schema" than the other way around :(

It was an expensive SVG, but it did a good job.

The bike is an actual bike with a diamond frame.

Now add irrelevant facts about cats and see if it can still draw it.

How much would it cost at list-price API pricing?

I don't have access but I wonder whether a dog on a jetski would be nearly as good