2 months ago
Self Hosting A Firefox Sync Server
In one form or another, Firefox Sync has been around for years (in fact, it's not _that_ far off being decade**s**...).
It allows you to share tabs, history and bookmarks between browsers on different devices (though, frustratingly, it doesn't sync settings or extension config).
Although sending that data via a 3rd party server might sound concerning, Mozilla are unable to see the content that's being synced because the data is end-to-end encrypted.
All the same, I like to self host where possible (and, frankly, needed something to keep me out of trouble), so decided to look at the feasibility of self-hosting a sync server.
This post talks about the process of setting the sync server up: it's been tested as working with Firefox, Waterfox and Firefox Mobile.
It should work with other fireforks too.
* * *
### Using an Upstream Project
I didn't have to look far to find an implementation of the sync server, because Mozilla publish theirs.
Unfortunately, they _don't_ publish much in the way of documentation on how to run it (to be fair, `syncserver-rs` is a Rust re implementation of their original which was pretty well documented).
Luckily for me, though, someone had already done the hard work and developed a `docker-compose.yml` to stand up the service and it's dependencies.
* * *
### First Deployment
There _were_ a few things in the `docker-compose` that I didn't like (particularly the creation of two separate MariaDB instances) but as the repo hadn't been touched in nearly a year I decided to start by deploying without modification to ensure that it still actually worked:
git clone https://github.com/porelli/firefox-sync.git
cd firefox-sync
./prepare_environment.sh
Enter FQDN for your Firefox sync server [firefox-sync.example.com]: fs.<my domain>
Enter full path for the docker-compose file [/home/ben/docker_files/config/firefox-sync]:
Listening port for syncstorage-rs [5000]: 8101
Max allowed users [1]:
Docker user [ben]:
I _did_ , however, change the volumes in `docker-compose` so that bind mounts were used instead of named volumes:
diff --git a/docker-compose.yml b/docker-compose.yml
index f9aa22f..e995788 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -45,7 +45,7 @@ services:
timeout: 5s
retries: 3
volumes:
- - syncstorage-db:/var/lib/mysql
+ - /home/ben/docker_files/files/firefoxsync/syncstorage-db:/var/lib/mysql
restart: unless-stopped
tokenserver_db:
@@ -64,7 +64,7 @@ services:
timeout: 5s
retries: 3
volumes:
- - tokenserver-db:/var/lib/mysql
+ - /home/ben/docker_files/files/firefoxsync/tokenserver-db:/var/lib/mysql
restart: unless-stopped
tokenserver_db_init:
@@ -84,7 +84,3 @@ services:
MAX_USERS: ${MAX_USERS}
DOMAIN: ${SYNCSTORAGE_DOMAIN}
entrypoint: /db_init.sh
-
-volumes:
- syncstorage-db:
- tokenserver-db:
I started the containers:
docker compose up -d
They all came up without complaint.
* * *
### DNS and Reverse Proxy
Next, I needed to set up DNS and configure my reverse proxy.
I pointed `fs.<my domain>` at the box hosting the reverse proxy and then added some Nginx config:
server {
listen 80;
root /usr/share/nginx/letsencryptbase;
index index.php index.html index.htm;
server_name fs.example.com;
location / {
return 301 https://$host$request_uri;
add_header X-Clacks-Overhead "GNU Terry Pratchett";
}
location /.well-known/ {
try_files $uri 404;
}
}
I invoked `certbot` to acquire a SSL certificate:
certbot certonly \
--preferred-challenges http \
--agree-tos \
--email $MY_EMAIL \
--webroot \
-w /usr/share/nginx/letsencryptbase \
--rsa-key-size 4096 \
-n \
-d "fs.example.com"
(actually that's only partly true - I have a wrapper which invokes `certbot` for me).
Once `certbot` had acquired a certificate, I added some more Nginx config:
server {
listen 443;
root /usr/share/nginx/letsencryptbase;
index index.php index.html index.htm;
server_name fs.example.com;
ssl on;
ssl_certificate /etc/letsencrypt/live/fs.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/fs.example.com/privkey.pem;
ssl_session_timeout 5m;
# Handle ACME challenges etc
location /.well-known/ {
try_files $uri 404;
}
location /{
proxy_pass http://192.168.13.125:8101;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
proxy_read_timeout 120;
proxy_connect_timeout 10;
gzip off;
add_header X-Clacks-Overhead "GNU Terry Pratchett";
client_max_body_size 10M;
# Let's not open this to the world
include '/etc/nginx/conf.d/auth.inc';
}
}
* * *
### Configuring Browsers
#### Desktop
Configuring a desktop browser to use a custom sync server is, on the face of it1, quite simple:
* If you're already signed in, sign out
* Go to `about:config`
* Search for `identity.sync.tokenserver.uri`
* Update from `https://token.services.mozilla.com/1.0/sync/1.5` to `https://<your domain>/1.0/sync/1.5`
* Open the main menu
* Click option to sign in
* Sign in with existing Mozilla sync credentials
If you tail the sync container's logs you should see something like this
Jan 03 12:39:17.275 INFO {"ua.os.family":"Linux","ua.name":"Firefox","ua":"140.0","ua.browser.family":"Firefox","metrics_uid":"59319a658b89f87d5120c4a0f559be2c","ua.browser.ver":"140.0","ua.os.ver":"UNKNOWN","uri.path":"/1.0/sync/1.5","token_type":"OAuth","uid":"<redacted>","uri.method":"GET","first_seen_at":"1767443957267"}
* * *
#### Android
Firefox Mobile (or Waterfox on Android) requires a slightly different approach
* If you're already signed in, sign out
* `Menu` -> `Settings` -> `About Firefox`
* Tap the logo 5 times, you should get a notification `debug menu enabled`
* Go back to `Settings`
* At the top should be `Sync Debug`. Tap it
* Tap `Custom Sync Server`
* Enter `https://<your domain>/1.0/sync/1.5`
* Tap `OK`
* An option to stop Firefox will appear, tap it
* Re-open Firefox
* Sign back in to sync
* * *
### Problems
It _mostly_ seemed to work: when I signed into a new browser install, it started installing extensions etc.
However, `Tabs from other devices` stayed resolutely empty.
After poking around a bit and getting nowhere, I searched the web and found this ticket on the original project - sync broke back in June and has (apparently) stayed that way since.
So, I updated my `docker-compose.yml` to pin to the tag published prior to the first broken one:
services:
syncstorage:
image: ghcr.io/porelli/firefox-sync:syncstorage-rs-mysql-0.18.3-20250526T064429-linux-amd64
The container restarted cleanly, but my browsers looked unhappy: the sync wheel span and span with no logs showing up in the container.
I decided to reconnect them, so I signed out of all of the browsers and then signed back in.
Tab syncing started working between my laptops!
* * *
#### Which Server?
Tab syncing was working, but shouldn't have been.
Although the two browser installs were _clearly_ communicating, my sync server wasn't generating any logs.
On one of the browsers, I went back to `about:config` and found that the setting had reverted:
I was initially a little confused and concerned by this. Had Mozilla screwed something up which meant that the setting wouldn't persist?
The cause turned out to be simpler than that: when you sign out of Sync, it resets this value to the default.
It turns out that that's not the only time that it can get overridden.
* * *
#### Firefox Mobile: But You Said...
At this point, I was almost ready to give up.
I decided to leave the desktop browsers using Mozilla's sync service and opened Firefox mobile in order to cut it back over.
I went through the same process as before:
* Signed out of sync
* Tapped the logo 5 times
* Went to `Sync Debug` and tapped `Custom Sync Server`.
It had originally been blank, so I blanked it and hit `OK` before restarting Firefox.
When it came back up, I signed back into sync but my sync server's logs showed Firefox Mobile continuing to connect to it.
I went back in and updated the custom URL to `https://token.services.mozilla.com/1.0/sync/1.5` before restarting again.
Signing back in, once again, led to logs on my sync server.
I rebooted the phone, no change.
Whatever I did, the phone continued to try and use my sync server.
Amongst the logs, though, I noticed that it was now successfully submitting tab information:
{
"ua":"145.0.2",
"ua.name":"Firefox",
"ua.os.family":"Linux",
"ua.browser.ver":"145.0.2",
"uri.path":"/1.5/4/storage/tabs?batch=true&commit=true",
"uri.method":"POST",
"ua.os.ver":"UNKNOWN",
"ua.browser.family":"Firefox"
}
So I decided to give things one more shot and point the laptops back at my sync server.
It worked!
* * *
### Additional Notes
Although this moves sync history onto infra that I control, it's not a complete solution: authentication is still performed using a Mozilla account.
It _is_ possible to self-host Mozilla's auth servers and there _are_ projects which simplify that process, but it's really a couple of steps further than I needed.
Speaking of authentication, by default the sync server is quite promiscuous: it'll handle syncing for anyone with a valid Mozilla account.
The `firefox-sync` project worked around that by introducing the `MAX_USER` environment variable. It's set to 1 in the `docker-compose.yml` so, once you've registered, it shouldn't allow anyone else to do so.
All the same, though, it would be prudent to enforce some kind of access control at the reverse proxy (for mine, access is limited to the LAN and my tailnet).
* * *
### Backing Up The Data
The data that's synced is encrypted, so backups are only _really_ useful for restoring the sync server if something happens to it.
We can pull credentials from the environment file and then use `mariadb-dump` to dump a backup of the databases:
# Load creds etc
source .env
# Backup the sync server
docker exec -it \
firefox-sync-syncstorage_db-1 \
mariadb-dump \
-u "$MARIADB_SYNCSTORAGE_USER" \
--password="$MARIADB_SYNCSTORAGE_PASSWORD" \
"$MARIADB_SYNCSTORAGE_DATABASE" | gzip > sync_storage-`date +'%Y%M%d'`.sql.gz
# Backup the token server
docker exec -it \
firefox-sync-tokenserver_db-1 \
mariadb-dump \
-u "$MARIADB_TOKENSERVER_USER" \
--password="$MARIADB_TOKENSERVER_PASSWORD" \
"$MARIADB_TOKENSERVER_DATABASE" | gzip > tokenserver_storage-`date +'%Y%M%d'`.sql.gz
We can also, out of idle curiousity, confirm that data does seem to be encrypted.
First, we connect to the sync server database:
docker exec \
-it firefox-sync-syncstorage_db-1 \
mariadb \
-u "$MARIADB_SYNCSTORAGE_USER" \
--password="$MARIADB_SYNCSTORAGE_PASSWORD" \
"$MARIADB_SYNCSTORAGE_DATABASE"
Then we query an item
select batch,payload from batch_upload_items limit 1;
Which results in something like this
As you'd hope, base64 decoding the ciphertext leads to a seemingly meaningless binary blob.
* * *
### Conclusion
Although this isn't something that the average Firefox user is going to need (or want) to do, the sync server is relatively easy to get up and running.
There are some _definite_ oddities in Firefox's config behaviour, so moving away from default settings does bring increased potential for confusing issues down the line.
But, having the data sync onto a system that I control means that I can do mundane things like _back it up_. It also, for what little it seems worth, means that my browser history isn't sat in a database which _might_ later be subject to Harvest Now, Decrypt Later.
Most of all, though, it's given me something fun to fiddle about with this afternoon. At some point, I'll likely look at consolidating the two MariaDB servers into a single instance and maybe lift shift the entire stack into my K8S cluster.
* * *
1. Should have known it's never that simple! ↩
Self Hosting A Firefox Sync Server
Author: Ben Tasker
www.bentasker.co.uk/posts/documentation/linu...
#firefox #mozilla #selfhosting #server #syncing
0
0
0
0
