Running several servers (pools)
A pool is two or more DartRelay servers that share one database and sit behind a load balancer under one address. This page explains what a pool is for, what it does not do, how to add servers to one, and how to take a server out for maintenance without interrupting people.
What a pool is
Most installations need one DartRelay server. You might add more when:
- You need more capacity than one server comfortably provides, and want new sign-ins spread across several.
- You want maintenance headroom: the ability to take one server out of service for Windows updates or a DartRelay upgrade while the others keep accepting new sign-ins.
Every server in a pool reads and writes the same database, so they share one configuration: the same Resources, users, themes, tenants, licence and settings. Change something in the console of any server and every server uses it. People reach the pool through one address on your load balancer, which decides which server handles each new visitor.
What a pool does not give you
A pool is capacity and maintenance headroom. It is not failover, and it does not make DartRelay highly available.
- Sessions do not move between servers. An application or desktop runs through the server that started it, for its whole life. If that server stops, its sessions end. The user can launch again, and the load balancer sends them to a server that is still running, but what was open is not carried across.
- Pass-through passwords are held by each server separately. Resources that sign in to Windows with the password the user typed at the portal use a copy kept on the server where they signed in. It is not shared with the other servers.
- The database is shared, so it matters. No server can serve anyone without it. Protect it as you would any business-critical database.
- Every server must run the same DartRelay version. Servers on different versions count seats differently and can disagree about who holds what. Upgrade them all together. See Upgrading DartRelay.
What a pool needs
| Requirement | Why |
|---|---|
| A shared database on SQL Server, MySQL or PostgreSQL that every server can reach | All configuration, sessions and seat counts live there. See Databases and migration. |
| A load balancer in front of the servers | Gives people one address, terminates HTTPS, and keeps each session on its server. See Load balancer and ADC integration. |
| The same DartRelay version on every server | See above. |
| Every server able to reach your Windows hosts | Any server may be asked to start any Resource. |
Adding a server to a pool
Start with one working, activated installation that already uses a shared server database. Then, for each additional server:
- Run the installer on the new server. When it asks how the server is licensed, choose Another server for an existing installation (join a pool) ("This server shares a database with the others").
- Give it the details of the existing installation, including a join token from that installation. A join token can be used once and expires after 24 hours. The installer checks what you enter with the existing installation before anything is installed, and stops with a message if it would not work.
- Finish the installation. The new server joins the shared database. It does not need its own product key: it reads the licence from the database.
- Add it to your load balancer. Add the new server's address and port to the load balancer's server group, with the same health check as the others.
- Check it. Open the pool's address, sign in and launch something. In the browser's developer tools, the response header
X-DartRelay-Nodeon the page request names the server that answered.
Anything specific to one server, such as its identity in the pool, its database connection and whether it starts drained, is set by the installer in that server's own configuration file. Everything else is shared through the database. See Server configuration file.
What is shared and what belongs to one server
| Setting | Where it lives |
|---|---|
| Resources, hosts, users, groups, themes, tenants, security settings | Shared database: change once, applies everywhere. |
| Load balancer settings: trusted proxies, the appliance sign-in options, public URL, server name header, header sign-in and the sign-out address | Shared, on System → Load balancer. A change reaches every server within about 30 seconds. |
| Licence | Activated once; every server reads it from the database. |
| Server identity in the pool, pool membership, the certificate that protects the pool's shared keys, start drained, database connection | Each server's own configuration file, written by the installer. |
A value set in a server's configuration file overrides the console for that one server. That is meant as a break-glass measure only: a field set this way shows a set on this server marker on the Load balancer page and cannot be edited there. Keep load balancer settings in the console so every server behaves the same.
Taking a server out for maintenance (draining)
Draining tells the load balancer to stop sending new visitors to a server while leaving everyone already on it alone. When the last session on it ends, you can update or reboot that server without interrupting anyone.
DartRelay reports its state at /health/ready. A server answers 200 when it is ready for new work, and 503 when it is draining or cannot reach its database or remoting service. Your load balancer's health check should ask that address; see what any load balancer must do.
- Drain the server. On System → Load balancer, use Drain for the server you want to take out.
- Wait for the load balancer to notice. Its next health check gets 503 and it marks the server down. New sign-ins now go to the other servers.
- Wait for sessions to finish. Watch Monitoring live sessions or the Server column on System → Seats until nothing is left on that server, or ask the remaining users to save and sign out.
- Do the maintenance.
- Put the server back. Undo the drain on System → Load balancer. The next health check returns 200 and the load balancer starts sending visitors to it again.
Draining only works if your load balancer actively checks /health/ready and leaves existing connections alone when a server is marked down. Open-source nginx does neither by default; see the nginx section of Load balancer and ADC integration for what to do instead.
Licensing and seats in a pool
- Activate once. One server holds the activation and publishes the licence terms to the shared database; the others read them from there and enforce the same seat limit.
- Seats are counted across the whole pool. A device with sessions on two servers is still one seat. If a server stops, its devices stop counting shortly afterwards.
- If the activated server goes quiet, the other servers stay licensed for 7 days from the last refresh. After a day without a refresh the licence strip warns, naming the server and the date. Bring the activated server back, or activate on another server in the pool.
See Licensing and Relay Seats for the seat rule itself.
When a request reaches the wrong server
A session's connection must reach the server that started it. If the load balancer sends it elsewhere, DartRelay refuses it rather than failing silently, and the user sees a message that the session is running on another server. This almost always means the load balancer is not keeping each visitor on one server; check its persistence (stickiness) setting.
Worked example
An accountancy firm has 120 staff and two DartRelay servers, relay-a and relay-b, behind a load balancer at relay.example.com, sharing a SQL Server database. On Tuesday evening the administrator needs to install Windows updates on relay-a. At 6 pm they drain relay-a. New sign-ins all go to relay-b. By 7:30 pm the last session on relay-a has ended; the administrator installs the updates, reboots, and undoes the drain. On Wednesday morning both servers take new sign-ins again. Nobody's work was interrupted, but if relay-a had failed outright at 6 pm instead, the people on it would have lost their open sessions and launched again on relay-b: that is the difference between a pool and failover.
If something goes wrong
Users see "running on another server"
The load balancer is not keeping each visitor on one server. Turn on cookie persistence (preferred) or source-address persistence. See what any load balancer must do.
Single sign-on from the load balancer works on one server only
A server's configuration file is overriding a shared setting. Open System → Load balancer on each server and look for fields marked set on this server.
Draining has no effect
The load balancer is not checking /health/ready, or it treats a down server by cutting its connections. Check its health monitor and its member-down behaviour.
Seat numbers look different on different servers
The servers are probably running different versions. Upgrade them all to the same version.
Every address in the audit log is the load balancer's
The load balancer is not in Trusted proxies, so DartRelay does not believe the client address it forwards. See Trusted proxies.
Related pages
Load balancer and ADC integration
What the load balancer must do, with configuration examples.
Databases and migration
Choosing a database a pool can share.
Licensing and Relay Seats
How seats are counted.
Upgrading DartRelay
Keep every server on the same version.
Sharing one licence
For separate installations rather than a pool.