The Story
For reasons unknown to me, Debian's NTP daemon only works on udp port 123, even when operating as a client.
This is a problem because my network configuration does not allow incoming packets on this port, thus preventing my raspberrySeed (raspberry pi running Debian, deluge, flexget) from syncing, causing it to slowly drift away from the real world.
To solve the issue, I did two things:

The Solution

1. Turn Gargoyle into an NTP server

Since Gargoyle is already syncing from outside, I only had to tell it to act as a server.
First, ssh into your gargoyle setup.
Edit the file /etc/config/system, and under timeserver, change the option enable_server to '1'.

After editing the file, restart the service using

/etc/init.d/sysntpd restart

And ensure the argument -l was added to the ntp daemon command line, using this code:

ps | grep ntp

2. Tell debian to ask Gargoyle:

ssh to your debian machine, and edit the file /etc/ntp.conf.
Comment out all of the server entires, and only leave one pointing to the name/IP address of Gargoyle:

After that, restart the ntp daemon by running

sudo service ntp restart

Wait a couple of minutes and use this to test everything is OK:

ntpq -p

You should see one entry for your Gargoyle router, with the other fields making sense (not all zeroes):

Enjoy your new time configuration!