A webhook callback from a render service should be handled as a notice that something changed, not as a final answer. Check that it really came from the service, acknowledge it quickly, do the heavy work somewhere else, and be ready to see the same notice twice or two notices out of order. Do those four things and a production application can rely on callbacks without being hurt by them.
One thing needs to be said before anything else. Our public pages for Templater, QUE and Projector do not document a webhook endpoint, and we are not going to describe one that we cannot point to. So this article has two jobs. The first half is how to build the receiving end of a callback from any render service, which is general practice and applies to whatever service your application talks to. The second half covers the notification routes our own tools do document, and how to get the same result from them.
The reader we have in mind is a backend developer at a software company that lets its users order a personalized video. A user fills in a form, the application sends a render request to a service, and some minutes later the service calls back to say the file is ready. The application then has to update the order, tell the user and store the result. That sounds like three lines of code. In production it is where most of the incidents live.
Treat the callback as a hint, and verify it
A webhook is a request that arrives at an address you published, from a sender you cannot see. Anyone who learns the address can send something to it. So the first rule is that the contents of a callback are claims, not facts.
The first check is authenticity. A well-designed service gives you a way to prove a message came from it, usually a signature computed from the body and a secret you share, or a secret value included in the request. Verify that before you read anything else, using the method in the service’s own documentation, and reject anything that fails with a plain error. Compare the signature against the raw body exactly as it arrived. A common mistake is to parse the body, change it in some way, and then fail to match the signature because the bytes differ.
The second check is meaning. Even a genuine callback can be stale, so do not let it write state you cannot take back. If the service offers a way to ask for the current status of a job, use the callback as a trigger to ask. Look up the job yourself and act on the answer you receive. This costs one extra request and it removes a whole class of problems, because you no longer depend on the order in which messages arrived or on whether the body was altered along the way. When a service does not offer a status lookup, treat the payload with more care. Match it against the order you already have, and refuse anything that refers to a job you never started.
The third check is scope. A callback should only be able to move an order forward along the path you expect. A job that your system records as delivered should not be moved back to rendering because a late message arrived. Write the allowed transitions down. Waiting to rendering. Rendering to delivered or to failed. Failed to waiting only when a person or a retry rule says so. Any message that asks for a move outside that list gets logged and ignored.
The last piece is what your endpoint reveals. Keep the address hard to guess, but do not rely on that. Do not echo the payload back, do not put secrets in the response and do not give a detailed error to a caller who failed authentication. Someone probing your endpoint should learn nothing.
Answer fast, work later, and expect repeats
The sender is waiting on your response, and it will not wait long. Services differ in how long, and we do not know yours, so read the documentation and find out. The principle holds regardless. Do the least amount of work needed to accept the message and then answer. Everything else belongs in a queue.
In practice that means your endpoint does three things. It verifies the request. It writes the message, as received, to durable storage or to a job queue. And it returns a success response. A separate worker picks the message up and does the real work, which is updating the order, fetching the finished file, notifying the user and anything else. If the worker fails, the message is still there and can be tried again without the sender being involved.
The reason to separate the two is practical. If you do the slow work inside the request and it takes too long, the sender decides you did not receive the message and sends it again. Now you are halfway through processing the first copy while the second arrives, which is the worst possible moment to be doing anything twice.
That brings up the thing every callback consumer eventually learns, which is that messages arrive more than once. Services retry when they do not get a clean answer. Networks duplicate requests. People replay events from a dashboard to test. Plan for it from the start by making handling idempotent, meaning that processing the same message a second time produces the same result as the first. The usual tool is an identifier. Services typically attach an event identifier or a job identifier to each callback. Record the identifiers you have handled, and when you see one again, acknowledge it and do nothing. If the service does not give you an event identifier, derive one from the job identifier and the status it reports, and use that.
Here is how the double-handling goes wrong in real life. A user orders a video. The render finishes and the service calls your endpoint. Your handler updates the order, then sends the user an email with the download link, then stores the file. Somewhere in that sequence the database is slow, so the response is late, and the service sends the same notice again. The second copy arrives while the first is between the email and the storage step. Now the user has two emails, one of them pointing at a file that has not finished saving, and your order record has been written by two workers at once. Nothing in the code looks wrong. Every line did what it was told. The fault is that the design assumed each message arrives once. A handled-identifier check at the front door, and a queue behind it, would have turned the second copy into a quiet no-op.
Order is the second thing you cannot count on. A message saying the job finished can arrive before a message saying it started, particularly when you have several workers or the sender retries one while sending the next. Do not build logic that assumes sequence. Use the allowed transitions from the previous section, and if a message would move an order backward, ignore it. Where timestamps are available, compare them. And again, where you can ask the service for the current state, do that and let it settle the argument.
Finally, decide in advance what happens when your own system is down. A sender will usually retry for a while and then give up. You cannot tell how long, and we cannot either, so check your service’s documentation for its retry behavior. Whatever it is, the window is finite, and a message can be lost on the way. That leads to the next point.
Plan for the callback that never arrives
The honest limit of any webhook design is that a notification that does not arrive leaves no trace. There is nothing to log, no error to catch. Your application simply never learns that a video finished. A production system needs a second path that does not depend on the first.
That path is reconciliation. On a schedule, find orders that have been in a rendering state for longer than they should be, and ask the service about each one directly. If the job is finished, update the order exactly as a callback would have. If it failed, record why. If the service has no record of it, mark the order failed and tell somebody. This can run every few minutes or every hour depending on how fast your users expect an answer. We are not going to suggest a number. Choose it from the point at which a user would start to wonder, and make it shorter than that.
With reconciliation in place, the callback becomes an optimization. It makes the common case fast, and the sweep catches the rest. That is a healthy arrangement, because you can now turn the webhook off for an hour during maintenance without losing anything. The orders wait, and the sweep picks them up.
Two small additions are worth making. Keep a record of every callback you received and what you did with it, including the ones you rejected. When a user says their video never arrived, you can then say whether the message came, whether it passed verification and what happened afterward. And put a limit on how many times the same order can be pushed through the same step by any path, so that a mistake cannot become a flood.
What our tools give you instead, and the bottom line
Since our own pages do not describe a webhook, here is what they do describe for finding out that a render finished, and how each one maps onto the ideas above.
Templater runs event scripts. Our documentation says you can run shell scripts or use the Templater API to create and run ExtendScripts that listen for events Templater broadcasts and hook into its processes. Our spot check page gives one example. You can pass the absolute path of a spot check to a script that is registered for a supported event. Our FAQ also notes that if a script reads no argument values, the arguments are most likely not being sent correctly, so check that first. An event script sits on the machine that renders, so it is the closest equivalent to a callback you control. It can do what a webhook handler would do. It can write a status back to your data source, call your own application’s address with the job identifier, or send an email or text message. Treat that address like any other callback endpoint and apply everything in the first two sections to it.
Projector covers delivery. Our pages say it retrieves metadata from the streaming platform and writes it back to your data source, and our post on Vimeo interactive video describes it monitoring the status of renders and uploads from the data source. If your application already reads that data source, it learns about a finished upload by reading a row, and no message has to be received at all.
QUE is the third route. Jobs belong to campaigns and the QUE Dash shows each job’s status. The QUE API is how your application creates campaigns and jobs. And the built-in Zapier integration includes a pre-made Zap that posts a Slack notification when QUE jobs are updated. That is a notification triggered by a job update, which is a different thing from a webhook endpoint you could point a custom application at. If that is what you need, ask us how to reach it, and read our documentation for the current options before building.
Notice that all three of these are mostly pull, not push. Your application reads a status from a data source or a dashboard, or runs a script on the render machine. That is not worse. Pulling on a schedule is exactly the reconciliation sweep described above, and a system that only pulls has no lost-message problem to begin with. The cost is delay, since you learn about a finished job at the next check instead of the moment it ended.
There is one more reason to prefer the pull route when the data is important. A system that reads a status has a natural place to put its own checks. It can compare the reported state with what it expected, it can read the same row repeatedly without side effects, and it can be tested by pointing it at a sample sheet. A system that receives pushes has to be tested by sending it messages, and the messages you forget to test are the ones that arrive in production. That is not an argument against callbacks. It is an argument for keeping the logic that reads a status and the logic that receives a notice as the same function, so that both routes always do the same thing and you only have to get it right once.
Our example developer, then, has a choice. If her render service offers signed callbacks, she builds the receiver as described. Verify, store, answer, process later, treat repeats as normal, and run a sweep behind it. If she is using our tools, she lets an event script write the result to the data source and reads it from there on a short schedule, and she adds the script’s own call to her application only if she wants speed. Either way the same rules apply. Do not trust the message more than it deserves, do not make the sender wait on your slow work, and do not let a missing notice mean a missing video.
The short version is this. A callback is a courtesy from a service that already knows the answer. Verify it, accept it quickly, handle it as many times as it comes, and keep a sweep running for the day it does not come. Do that, and a production application can use webhooks and still sleep.
Developers usually ask us a few things before they build a callback receiver, and a few more about what our own tools offer.
Frequently Asked Questions
Does Dataclay offer a webhook for finished renders?
Our public pages do not document a webhook endpoint for Templater, QUE or Projector, so we do not describe one. What they do document is event scripts in Templater, Projector writing metadata back to your data source, the QUE Dash job status, and a Zapier Zap that posts a Slack notification when QUE jobs are updated. Contact us if you need something beyond those.
How do I know a webhook really came from the render service?
Use the verification method in the service’s own documentation, usually a signature computed from the request body and a shared secret. Check it against the raw body exactly as it arrived, before you parse anything. Reject failures with a plain error that reveals nothing. A callback that passes is still only a claim, so confirm the job’s status with the service when it offers a lookup.
What should my webhook endpoint do before it responds?
Very little. Verify the request, write the message to durable storage or a job queue, and return a success response. Do the real work in a separate worker. If you do slow work inside the request, the sender may decide you never received it and send it again, which starts a second copy while the first is still running.
Why does the same callback arrive twice?
Services retry when they do not get a clean answer, networks duplicate requests, and people replay events while testing. Treat repeats as normal. Record the event or job identifier you have handled, and when you see it again, acknowledge it and do nothing. If there is no event identifier, build one from the job identifier and the reported status.
What if a callback never arrives?
A lost notification leaves no trace, so you need a second path. Run a sweep on a schedule that finds orders stuck in a rendering state and asks the service about each directly. Update them exactly as a callback would. With a sweep behind it, the callback only makes the usual case faster, and nothing is lost when your endpoint is down.
Can a Templater event script stand in for a webhook?
It can play the same role on the render machine. Our documentation says event scripts listen for events Templater broadcasts, and that scripts can write results to a data source or send an email or text message. If a script calls your own application, treat that address like any callback endpoint and apply the same verification and idempotency rules.
