FeathersJS REST queries with null

I have been working on a FeathersJS project which was using realtime on the web client, during a performance review we had been looking at what can be removed that wasn't necessary in order to reduce the core / vendor bundle sizes. As realtime wasn't being used for any realtime messages or updates this was a relatively obvious place to look.

Having been using FeathersJS for a number of years now I thought that this would be a relatively straight forward change, even though I had yet to swap client transports on a project myself and had in fact been using realtime for the majority of the projects I have been involved in. The code change itself was as expected pretty simple, install the new client libraries for REST and install node-fetch as this was a Server Side Rendered application.

The problems begin

However after restarting the application and visiting a few pages locally it became apparant that something wasn't right. Pages were not loading properly, so I began to investigate.

Array issues

We had quite complex queries which had lots of deeply nested parameters which would then be used within a hook to make associations which worked perfectly fine with websocket queries, but was totally broken when using REST. After looking into how FeathersJS parses the query string I found that it was using the querystring package which had an option to increase how deep it would parse an nested string. It will by default only parse up to 5 children deep, which was a problem as we had queries which went deeper. This is what would happen if you had a query with more than 5 children deep:

1const input = 'a[b][c][d][e][f][g][h][i]=j'
2const output = {
3    a: {
4        b: {
5            c: {
6                d: {
7                    e: {
8                        f: {
9                            '[g][h][i]': 'j'
10                        }
11                    }
12                }
13            }
14        }
15    }
16}

Array solution

The solution is highlighted within faqs whereby you can set a custom qs instance with your own options. This allows you to customise how deep you would like the querystring to be parsed:

1// Update limits for `qs` to allow for deeper
2// parsing of objects specifically from REST
3app.set('query parser', function(str) {
4    return qs.parse(str, {
5        arrayLimit: 100,
6        depth: 20,
7        parameterLimit: 2000
8    });
9});

null issue

The next issue which also appears within the faqs is that a null value when in a query being sent via REST is not being handled correctly, it will appear as an empty string. This doesn't work when using Sequelize with postgres as a null value would typically be turned into a 'property' is not null query, which ends up being 'property' != ''.

null solution

In order for null to be handled correctly we must look through the qs library API documentation. We can see that there is an option to allow for strictNullHandling which we can add to the server's query parser like we set above.

1// Allow for strict null handling
2app.set('query parser', function(str) {
3    return qs.parse(str, {
4        arrayLimit: 100,
5        depth: 20,
6        parameterLimit: 2000,
7        strictNullHandling: true
8    });
9});

The issue now is how do we set how the qs library behaves on the client when it calls qs.stringify on the query object? Well currently this isn't possible, there isn't a way to be able to pass options to the qs.stringify through FeathersJS rest-client, nor is there a way that will allow you to extend and customise how the querystring is created.

But fear not, for I have a PR which will remedy this issue. I had originally created a PR for the ability to add options to this effect, but after a short discussion and additional information from David the core developer on FeathersJS it was decided that allowing extensibility was favoured over options, so a new PR was made to make this possible. It will allow the following to be possible:

1import feathers from '@feathersjs/feathers'
2import rest from '@feathersjs/rest-client'
3import {
4    FetchClient
5} from '@feathersjs/rest-client'
6import qs from 'qs'
7const app = feathers()
8
9class MyFetchClient extends FetchClient {
10    getQuery(query) {
11        if (Object.keys(query).length !== 0) {
12            const queryString = qs.stringify(query, {
13                strictNullHandling: true
14            }) return `?${queryString}`
15        }
16        return ''
17    }
18}
19// Configure API url
20const restClient = rest('https://feathers-api.com')
21// Configure rest client
22app.configure(restClient.fetch(window.fetch, MyFetchClient))

With of these changes in places this allows for a much smoother transition between REST and realtime web clients for a FeathersJS API.