Background processing with Firebase Cloud Functions

Question

I recently started seemingly similar topic here, but I feel that maybe I implied too much in my question by asking how to implement something instead of asking how to solve specific problem. So here I go asking from different angle:

• third-party API (most possibly webhook) sends .csv file and .docx files (data and template) and sends response as fast as those files are uploaded (no waiting until those documents are processed)
• server merges that data and whenever the data is ready it sends to the user-specified endpoint response with download link
• I want to use Firebase products to achieve that
• it has to be compatible with typical automation tools like Zapier, Pabbly etc. (it just has to work like typical webhook)

In my previous question I got quite interesting answer to use PubSub (almost tried it, but got error while installing it), but I'm thinking - maybe there is some easier way to solve this?

question from:https://stackoverflow.com/questions/65847025/background-processing-with-firebase-cloud-functions

Answer 1

Like I wrote in my last comment to your other question, if you plan to send heavy files to Cloud Functions, bear in mind that the size limit for data sent to an HTTP Cloud Functions is 10MB (See doc). There is the same limit for the size of messages you can push to Pub/Sub. (See doc).

One approach would be to upload the files (data and template) to Cloud Storage and pass their references to the HTTP Cloud Function and, from this one, pass them in the payload of the Pub/Sub message (as explained in the other answer). Then in the Pub/Sub Cloud Function, you read the files from Cloud Storage.