@Robin
Typically you’re looking to join additional tables to your original query, but I guess it could also be used as a filter.
Lane said:
@Robin
Typically you’re looking to join additional tables to your original query, but I guess it could also be used as a filter.
So does that mean these additional tables have data from the User info?
Zeek said:
@Shai
Yeah, my first thought was the N+1 problem. The person might be able to reduce their database queries by a factor of 10 with a join.
Firebase works as a document store. Joins aren’t really a thing with it. NoSQL has its quirks.
@Shai
Honestly, I’m not sure why it’s reading so much. With 547 employees, I have fields like: Employee Number, Name, Position, Roster, Section, Start Date, Supervisor, id.
When I open the worker page, all workers are displayed, and I can edit or terminate them. Seems like that’s where most of the reads are happening.
Should I change the workers into one long field instead?
I like using Firebase, but I’m worried about overcharges once I switch to the blaze plan due to code inefficiencies.
The goal of this web app is worker management. There are three main pages - leave form, workers, and roster, with the roster showing the worker’s schedule displayed on an AG-Grid Table. The leave form lets me search for a worker and add their leave as well as a signed form. This works fine, but the roster page does not reflect it (it only shows the full roster). The worker page reads all workers, and I’m thinking the way I made each worker a document in a collection is causing inefficiencies.
Nope, just building this instead of using Excel because my Excel file is becoming too unwieldy now. I’m more than willing to spend out of pocket to make this easier for myself.
@Penn
That’s awesome to see non-devs using these tools.
If file size and number of queries are becoming a concern, you might want to explore SQL with a focus on indexing and optimized queries. That knowledge should help you avoid excessive computing costs.
@Penn
Are you making sure to save your first pull’s results in memory and not calling it every time the state refreshes? I’ve pulled thousands of rows in Firebase on the first load without hitting the limit.
@Shai
Just a heads up for those who aren’t aware, the Firestore dashboard counts against your quota. Trust me, I learned it the hard way.
@Shai
I faced this issue not long ago. Firebase can be a really powerful tool.
I even created a thread about it: https://www.forum.com/topic/backend-issues-from-a-frontend-dev/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button
What we did to improve ours was to define the logic to fetch data once on a base component and add a refresh button to avoid constant refetching.
Also, if you’re using pagination, you might want to check out Firestore’s .limit() function, though it might be a bit tricky to implement.
By the way, Query Cursors might be the ideal method to implement pagination with Firebase.
I’m not a Firebase user, but it seems you’re handling a trivial amount of data and users:
-
Read less data through a well-structured schema and interactions, and pagination if possible.
-
Index your data. Sometimes, less is more, or you’ll sacrifice write performance. Based on your usage, can you select key fields you frequently filter on?
-
Consider putting an in-memory cache layer between your database and application, like Redis or memcached.
On another note, I’ve never seen anyone use Firebase without regrets. You may be better off setting up your own database instance on a VPS. Managed database services are great for availability and upgrades, but can be expensive if your app isn’t making any money. You might want to manage your own for now.
For future projects, I’d advise against using Firebase or NoSQL databases. Focus on getting to know relational databases like Postgres.
Here are some options:
-
Stop using Firebase and pick a traditional SQL database you can manage and won’t lock you out when querying 600 records at once.
-
Implement a Redis cache in front of Firebase to avoid excessive database hits and read from the cache instead.
-
Maintain another document in Firebase with a list of all 600 workers, like a basic listing page without detailed info for each.
All these have pros and cons. But generally, if you’re querying 600 items constantly, you may need to rethink your app’s logic.
@Perry
The person who posted this is a beginner. These suggestions are likely more complex than they can handle at this point. They might be missing foundational concepts and don’t require a caching layer.
@Rory
This happens often in these discussions; solutions can be way too complex for the person’s actual needs. The top comment rightly points out that 600 reads per load is excessive, and they should reconsider their logic. The other two options are manageable in this scenario.
Here are a few suggestions:
-
When you look at the roster, you mentioned reading the entire database. Is this truly needed? What data do you really need for that view? Can you adjust your code to only pull the necessary data?
-
If the data isn’t frequently changing, caching it could help. Consider options like memcache, redis, apcu, or even storing it in a file.
-
A combination of both approaches would be ideal.
@Jay
The worker page is what’s causing all these reads. It shows all the workers we have, and these can be edited. Wouldn’t editing require reading the whole database again? I’m really new to all this. I’m an electrical engineer trying to learn something new.
@Penn
You could cache all the workers.
Then for every view, you pull from the cache, not the database.
When an edit occurs, refresh the cache. If that still leads to many reads due to frequent edits, consider updating both the database and the cache for each change. This can get tricky with relations since you’d need to ensure everything stays in sync.