Header image
banana4life

LDJam API misbehaving

posted by Unknown | 04 May 2021

I'm currently working on a new feature for the website of our LDJam team (https://banana4.life) that's using the api.ldjam.com in order to pull certain things. I'm only using /vx/node/get/..., /vx/node/feed/... and /vx/node/walk/....

While trying to fetch the feed of platform tags (the values of the "platform" dropdown when you add downloads to a game), I noticed a weird behavior in the API.

When I get the feed as one large request with the following link, I get the 89 unique node IDs I expect: https://api.ldjam.com/vx/node/feed/0/all/tag/platform?offset=0&limit=250

This can be reproduced with: curl -s 'https://api.ldjam.com/vx/node/feed/0/all/tag/platform?offset=0&limit=250' | jq -r '.feed[].id' | sort | uniq | wc -l

However if a get the feed in two chunks by doing the request first with offset=0&limit=50 and then with offset=50&limit=50, while still the same amount of node IDs, the list contains duplicates.

This can be reproduced with: curl -s 'https://api.ldjam.com/vx/node/feed/0/all/tag/platform?offset=0&limit=50' 'https://api.ldjam.com/vx/node/feed/0/all/tag/platform?offset=50&limit=50' | jq -r '.feed[].id' | sort | uniq | wc -l

The first command prints 89, the second command print's 57 (at least at the moment I tried this today).

I looked at the implementation of the API over at https://github.com/ludumdare/ludumdare to see what is going on. When I follow the code paths for the given argments (root=0, method=[all], type=tag, subtype=platform, offset=..., limit=50) and it seems to eventually end in a SELECT ... ORDER BY n.published DESC LIMIT 50 OFFSET ... query against the MySQL database, where n is the table with all the nodes.

From what I read about MySQL, OFFSET happens after ORDER BY, so the result should be consistent, however the behavior I see tells a different story.

Can anyone confirm the behavior and possibly even explain it?

Code path that's taken:

  1. Controller code behind /node/feed at the line it calls into the DB access code: https://github.com/ludumdare/ludumdare/blob/master/public-api/vx/node.php#L432
  2. The DB access code at the point where the DB query is executed: https://github.com/ludumdare/ludumdare/blob/master/src/shrub/src/node/node_feed.php#L514