OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Re: ways of reducing mysql load

From: Andre Tomt (andretomt.net)
Date: Mon Jul 19 2004 - 16:57:14 CDT


Len Conrad wrote:

> Make your MX front-end server different from your backend mailbox
> server).

This setup is made for plugging in $somewhat_large_num server in all
parts of the system. Front-end, outgoing-relay and incoming-mx is
already separated. Scalability is already been pretty much thought out.

> Export from your backend db to the MX box the data for relay
> domains, transport, relay_recips, which:
>
> 1. removes the real-time dependence of send/receiving mail upon the
> backend database (mail is accepted even if the backend db is down).
> isolation of the backend db from the MX by duplicating the db data is an
> extremely important scaleability and reliability feature.
>
> 2. totally eliminates db queries from the MX to the backend db, greatly
> speeding up MX operation, and solving the problems you have created. If
> your db is slow (or down for db maintenance), the MX is slowed, or down,
> to the db speed, which is poor system design.
>
> 3. avoids the DoSing of your backend db when the MX is DoSed. With your
> setup, high-volume attacks using unknown recipients (aka "daily
> operation") are passed straight through from the MX to the backend db.

Real-time replication of the databases to each server or a database
cluster is a part of the planned finished design - depending on what
proves to scale best. I'm just looking into ways to reduce the query
load to make the system even more resilent to outside disturbances.

A mitigating solution have already been brought up - transport_map and
relay_domains beeing exported to local lookup tables. This will drop the
database load quite a bit, even though recipient maps still has a
real-time dependency and thus uses the database directly. I'll integrate
that into the maintance system.

--
Cheers!
André Tomt