Electronic Mail Monitoring
Performance Monitoring
Service Level Reporting
Email SLA Statistics
Alarm Notifications
About Us
Customer List
Free Trial Download
Home |
The Question: What is the Quality of Our Email Delivery Services?
The Quality of Service Issue is summed up in: How long is it taking for our users to send and receive emails? This is a question that hangs out there 24 hours a day.
Who Wants To Know? And How Do We Play Nice About It?
Everybody wants to know. And so they ask the electronic messaging gurus. Actually they ask the managers of the gurus. And those managers are often in different regions, corporate divisions, all the way to affiliates in other countries. So we want to know in an unbiased way how long email deliveries are taking back and forth, 24 by 7, 365 days out of the year. Very often, these good people do not work for each other directly but they have the same common service goal: Speedy, reliable, and safe delivery of email and its related attached content.
How Do YOU Demonstrate Email Delivery Quality Of Service?
Email QOS is a matter of consensus and expectation. It is usually stated something like:
- All Email Will Be Delivered Within 10 Minutes
- Ninety Nine Percent of All Email Will Be Delivered in Five Minutes
- During Normal Business Hours All Email Will Be Delivered Within One Minute
Two Methods of Email Delivery Data Gathering
There are 2 methods to demonstrate QOS of Electronic Mail Delivery:
- Grab all mail server log data and run reports against the time stamps in these logs.
- Get a statistically valid subset of the message traffic and use that subset to create reports.
Method 2 is superior for a numbers of reasons.
Method 1: It's a lot of work for what you get.
- Clocks among servers are not synchronized well. Off by many seconds and very often minutes.
- Time zone differences are often ignored. Some servers are clocked in the wrong time zone for none other than historical reasons. Only minutes are counted. Negative elapsed times are dropped or, worse yet, counted in the average.
- Data gathered is point-to-point. Multiple hops are not reconstructed. How long is it taking on average to go through multiple hops from Omaha to Hong Kong? (Warren Buffet may want to know?)
- Data gathering in the event of backup routing usually ignored and certainly not tested well. And in a network failure the lack of delivery statistics for forensic analysis is not a good thing.
- This method requires a good deal of time to set up and maintain. It is costly.
- Very difficult to do across domains, affiliate companies, and countries. Nearly impossible regarding customers and vendors who also have their own private networks.
Method 2: It better represents what the Email End User is experiencing.
They are sets of samples taken at regular intervals. Each set is an email delivery route, from New York City-to-Rome, for instance. The NY/Rome Set would be comprised of regularly-spaced email test messages which travel with the production messaging stream just like any other user. A delay in the route is reflected in the delivery time.
So let's look at why this technique is preferable:
- Delivery elapsed times have a single time reference, the testing computer. As such they are accurate as possible.
- For any Email Test Set, the number of samples is large enough to realistically represent the total population of messaging traffic over this route. At one test every 30 minutes, a Monthly Service Level Report for NYC/Rome would be derived from 2 x 24 x 30 days or 1,440 samples. Email route testing sample sizes of this size are considered to accurately represent the population as a whole.
- Multiple hop sets such as Omaha to Hong Kong are measured at the end points. This is the user experience. While these times may be longer than just taking data between servers, it is what the user is experiencing and is more accurate.
- If something in the network fails and backup routing takes place, you get to see exactly how well that backup routing conforms to the stated Email Delivery Service Level.
- It is less costly because the entire operation can be automated. Even the reporting.
- You can test to the internet and outside vendors, customers, and groups whose equipment is also within a private network.
- This frequent sampling method is used just about everywhere to demonstrate quality.
- Pharmaceutical companies produce drugs where samples are gathered at regular intervals. The samples are subjected to tests of dosage correctness, mixture homogeneity, etc.
- QC personnel pull product from factory lines at regular intervals and test it for strength, defects, color, taste, bacterial content, etc.
- Automakers like Toyota do their own tests on sample sets of steel roll stock right at the their vendor's mill. They test to demonstrate how well it conforms to their minimum quality specifications.
- We all have experienced customer follow up via telephone on services done to repair our cars, fix our computers, re-connect our cable outages, etc.
These groups record and publish their results as representative of the larger population of product being produced, and the larger population of services being provided.
|
