Tony

Venus Uptime Report Date Average response time Uptime Downtime 1 240.972 ms 100.00% - 2 140.159 ms 100.00% - 3 258.525 ms 100.00% - 4 316.750 ms 100.00% - 5 256.384 ms 100.00% - 6 286.800 ms 100.00% - 7 232.733 ms 100.00% - 8 288.405 ms 100.00% - 9 215.641 ms 100.00% - 10 265.967 ms 100.00% - 11 202.006 ms 100.00% - 12 208.415 ms 100.00% - 13 192.477 ms 98.76% 17m 17s 14 182.107 ms 100.00% - 15 161.551 ms 99.69% 4m 28s 16 221.421 ms 100.00% - 17 337.353 ms 100.00% - 18 251.318 ms 100.00% - 19 251.552 ms 100.00% - 20 247.114 ms 100.00% - 21 381.241 ms 100.00% - 22 374.441 ms 100.00% - 23 354.905 ms 100.00% - 24 189.805 ms 100.00% - 25 185.949 ms 100.00% - 26 118.443 ms 100.00% - 27 158.910 ms 100.00% - 28 212.231 ms 100.00% - 29 176.874 ms 100.00% - 30 166.479 ms 100.00% - 31 131.780 ms 99.94% 49s Average Response Time: 232.539 ms Uptime %: 99.95% Total Downtime: 22m 34s Reponse Time Graph (Click for full size) [ATTACH]20[/ATTACH] Uptime Graph (Click for full size) [ATTACH]19[/ATTACH] The only down time this month related to the large DDOS experienced during the late hours that affected numerous providers in the area. But overall we easily reached a 99.9% uptime for the month.

Here's May the final month of me posting this as we'll have public uptime available for the July posting. Anyways here's the uptime reports for all our servers.

One other thing I forgot to mention I don't really advise doing it with awstats as there are exploits out there and such for it. The others to my knowledge just generate files so there isn't a big deal. I'd also make use of a .htaccess if you don't plan on having it 100% public as search engines tend to find everything.

There isn't a way to do this with a quick snap of the fingers in cPanel. However you can do this by making use of symbolic links. You'll need SSH to do this so if you do not have it or are unsure about this method we can do it for you. First you make a directory in your public_html folder called say stats Create Your Symbolic Link: ln -s /home/yourusername/tmp/webalizer stats Chmod Directories: /home/yourusername/tmp - chmod to 755 /home/yourusername/tmp/webalizer - chmod to 755 Of course with all of this changing "yourusername" with your actual account username.

This is to announce a scheduled power maintenance window for the electrical infrastructure of the sr04 facility. The maintenance is scheduled to begin on Monday, May 21st at 8:00pm CDT (GMT-5) and will last 12 hours until Tuesday, May 22nd at 8:00am CDT (GMT-5). The maintenance will be on the electrical infrastructure of sr04. This maintenance is not expected to be service impacting. Electricians and contractors will be on site throughout the maintenance window and after to perform the work and to monitor all affected systems. This maintenance window is not expected to be service impacting. Date: 5/21/2007

This issue was resolved via a ticket the problem was corrupted frontpage extensions which was fixed by reinstalling them.

Date: 5/7/2007 Start time (EST): 5:00am End time (EST): 7:00am Services affected: SoftLayer Internal Peering Location: cer01.dal01 / cer02.dal01 Duration: 2 hours SoftLayer engineers will be performing maintenance on internal peering circuits between the hours listed. While this will not be service impacting, there may be some latency related problems while routes reconverge.

Jupiter Uptime Report Date Average response time Uptime Downtime 1 N/A N/A N/A 2 N/A N/A N/A 3 N/A N/A N/A 4 N/A N/A N/A 5 N/A N/A N/A 6 N/A N/A N/A 7 N/A N/A N/A 8 N/A N/A N/A 9 N/A N/A N/A 10 N/A N/A N/A 11 N/A N/A N/A 12 N/A N/A N/A 13 N/A N/A N/A 14 N/A N/A N/A 15 N/A N/A N/A 16 N/A N/A N/A 17 N/A N/A N/A 18 N/A N/A N/A 19 N/A N/A N/A 20 224.864 ms 100.00% - 21 132.831 ms 100.00% - 22 156.153 ms 100.00% - 23 291.108 ms 100.00% - 24 255.024 ms 100.00% - 25 177.539 ms 100.00% - 26 160.908 ms 100.00% - 27 210.755 ms 100.00% - 28 114.108 ms 100.00% - 29 232.005 ms 99.87% 1m 48s 30 187.929 ms 100.00% - Average Response Time: 194.839 ms Uptime %: 99.99% Total Downtime: 1m 48s Reponse Time Graph (Click for full size) [ATTACH]17[/ATTACH] Uptime Graph (Click for full size) [ATTACH]18[/ATTACH] Jupiter was brought online late in the month it posted a good uptime the only outage was most likely just the web server rebooting and the monitoring picked it up. The monitoring system does checks in 1 minute intervals so if it picks up the service being down it is counted as a total of 1 minute minimum.

Mars Uptime Report Date Average response time Uptime Downtime 1 149.441 ms 100.00% - 2 129.789 ms 100.00% - 3 141.773 ms 100.00% - 4 127.077 ms 100.00% - 5 138.347 ms 100.00% - 6 121.486 ms 98.88% 16m 4s 7 124.616 ms 100.00% - 8 117.444 ms 100.00% - 9 150.897 ms 100.00% - 10 154.451 ms 100.00% - 11 150.984 ms 99.89% 1m 34s 12 115.573 ms 100.00% - 13 126.394 ms 100.00% - 14 132.046 ms 95.21% 1h 8m 15 124.827 ms 90.16% 2h 20m 16 136.220 ms 100.00% - 17 128.433 ms 100.00% - 18 150.162 ms 100.00% - 19 205.339 ms 100.00% - 20 199.453 ms 100.00% - 21 150.488 ms 99.85% 2m 4s 22 188.618 ms 100.00% - 23 245.251 ms 100.00% - 24 252.726 ms 100.00% - 25 215.145 ms 99.56% 6m 18s 26 165.400 ms 100.00% - 27 274.479 ms 99.07% 13m 10s 28 109.717 ms 100.00% - 29 309.464 ms 100.00% - 30 194.027 ms 100.00% - Average Response Time: 164.336 ms Uptime %: 99.40% Total Downtime: 4h 8m Reponse Time Graph (Click for full size) [ATTACH]15[/ATTACH] Uptime Graph (Click for full size) [ATTACH]16[/ATTACH] The server like the others was affected by the power outage. We also did a kernel upgrade late in the month. As for the other small outages I'm not sure about monitoring picked them up but they appear to be somewhat isolated and could have simply been apache being flooded with technicians solving the problem and blocking offending ip's.

Mercury Uptime Report Date Average response time Uptime Downtime 1 159.422 ms 100.00% - 2 115.578 ms 100.00% - 3 147.424 ms 100.00% - 4 121.439 ms 100.00% - 5 127.083 ms 100.00% - 6 155.528 ms 99.87% 1m 54s 7 114.271 ms 100.00% - 8 119.988 ms 99.67% 4m 42s 9 151.359 ms 100.00% - 10 139.147 ms 100.00% - 11 143.187 ms 99.84% 2m 19s 12 126.482 ms 100.00% - 13 116.788 ms 100.00% - 14 142.847 ms 94.94% 1h 12m 15 115.447 ms 90.16% 2h 20m 16 138.729 ms 100.00% - 17 143.390 ms 95.58% 1h 2m 18 158.103 ms 100.00% - 19 221.290 ms 100.00% - 20 137.586 ms 100.00% - 21 141.850 ms 100.00% - 22 169.329 ms 100.00% - 23 282.215 ms 100.00% - 24 195.543 ms 100.00% - 25 211.972 ms 100.00% - 26 201.946 ms 100.00% - 27 210.597 ms 100.00% - 28 110.940 ms 100.00% - 29 230.864 ms 100.00% - 30 165.229 ms 100.00% - Average Response Time: 157.186 ms Uptime %: 99.32% Total Downtime: 4h 44m Reponse Time Graph (Click for full size) [ATTACH]13[/ATTACH] Uptime Graph (Click for full size) [ATTACH]14[/ATTACH] The power outage affected this machine but unfortunately it also got hit by a large dos attack which caused the machine to crash. When rebooting it unfortunately the file system needed to be checked in case of corruption, this took an hour to complete.

Venus Uptime Report Date Average response time Uptime Downtime 1 131.996 ms 100.00% - 2 111.240 ms 100.00% - 3 143.644 ms 100.00% - 4 150.203 ms 100.00% - 5 129.246 ms 100.00% - 6 128.621 ms 100.00% - 7 143.324 ms 100.00% - 8 131.763 ms 100.00% - 9 153.875 ms 100.00% - 10 137.886 ms 100.00% - 11 114.887 ms 100.00% - 12 115.060 ms 100.00% - 13 113.470 ms 100.00% - 14 113.087 ms 95.22% 1h 8m 15 115.156 ms 89.79% 2h 25m 16 141.109 ms 100.00% - 17 115.080 ms 100.00% - 18 138.402 ms 100.00% - 19 181.812 ms 100.00% - 20 153.000 ms 100.00% - 21 137.502 ms 100.00% - 22 204.743 ms 100.00% - 23 229.461 ms 100.00% - 24 255.322 ms 100.00% - 25 220.109 ms 100.00% - 26 143.444 ms 100.00% - 27 240.524 ms 100.00% - 28 111.669 ms 100.00% - 29 206.633 ms 100.00% - 30 187.795 ms 100.00% - Average Response Time: 153.336 ms Uptime %: 99.49% Total Downtime: 3h 34m Reponse Time Graph (Click for full size) [ATTACH]11[/ATTACH] Uptime Graph (Click for full size) [ATTACH]12[/ATTACH] As stated at the start the only downtime relates to the server losing power which was out of our control.

Apollo Uptime Report Date Average response time Uptime Downtime 1 113.566 ms 100.00% - 2 115.074 ms 100.00% - 3 112.978 ms 100.00% - 4 126.135 ms 99.87% 1m 52s 5 129.114 ms 100.00% - 6 129.270 ms 100.00% - 7 116.015 ms 100.00% - 8 123.200 ms 100.00% - 9 146.009 ms 100.00% - 10 144.377 ms 100.00% - 11 123.792 ms 100.00% - 12 119.177 ms 99.75% 3m 34s 13 146.134 ms 100.00% - 14 118.226 ms 99.69% 4m 28s 15 122.527 ms 99.87% 1m 50s 16 119.553 ms 100.00% - 17 127.432 ms 100.00% - 18 149.720 ms 98.46% 21m 22s 19 203.870 ms 100.00% - 20 168.695 ms 100.00% - 21 141.747 ms 100.00% - 22 212.197 ms 100.00% - 23 260.297 ms 100.00% - 24 229.536 ms 100.00% - 25 248.282 ms 100.00% - 26 141.782 ms 100.00% - 27 262.005 ms 100.00% - 28 97.680 ms 99.87% 1m 48s 29 248.412 ms 100.00% - 30 180.190 ms 100.00% - Average Response Time: 155.900 ms Uptime %: 99.92% Total Downtime: 34m 54s Reponse Time Graph (Click for full size) [ATTACH]9[/ATTACH] Uptime Graph (Click for full size) [ATTACH]10[/ATTACH] There is a few couple minute outages which I'm unsure about the monitoring may have picked up a reboot of the web server. The 23 minute outage was unfortunately due to the server crashing which we rebooted the second we were notified by monitoring.

April was unfortunately a tough month at Devoted Host as our primary datacenter SoftLayer was running on generators due to the inclement weather in the area. When switching back to utility power a 2500amp breaker failed placing all our servers on UPS backup power. Unfortunately after 30 minutes machines started to go offline. After about 3 hours of work from SoftLayer electricians onsite power was restored to servers. I'd like to make things clear here things happen from time to time that are unexpected. Hardware does fail the best you can do is restore service as fast as possible. The 3 hour timeframe this was done in was significantly better than other datacenters in the past have replaced breakers of this size. So here is our uptime reports for all our servers it includes the no longer active Apollo as well as the now active Jupiter.

At 11:00PM EST, SoftLayer Datacenter lost power to all racks in SR02 and 5 racks in SR01, and one rack in SR03. This was due to a power transfer from generator to utility due to inclement weather in the DFW area. During the transfer, a 2500amp breaker failed placing all power on UPS backup power. After roughly 30 minutes on battery backup, power to the specified datacenters began to go offline. Electricians are onsite replacing the 2500amp main breaker with a cold swap spare kept onsite. We expect this situation to be resolved by 1:00AM CST. The emergency reponse team is onsite to bring servers and services affected back online when power is restored. Start time (EST): 23:00 Services Affected: Power SR02, SR03 (All Devoted Host shared, reseller servers except Apollo and dedicated customers) Device: Main ATS Switch Location: dal01 Duration: 3hr 25min End Time (EST): 04/14/2007 02:25

Mars Uptime Report Date Average response time Uptime Downtime 1 N/A N/A N/A 2 N/A N/A N/A 3 N/A N/A N/A 4 N/A N/A N/A 5 N/A N/A N/A 6 N/A N/A N/A 7 N/A N/A N/A 8 N/A N/A N/A 9 N/A N/A N/A 10 N/A N/A N/A 11 N/A N/A N/A 12 N/A N/A N/A 13 N/A N/A N/A 14 165.601 ms 100.00% - 15 210.648 ms 99.93% 58s 16 200.167 ms 100.00% - 17 132.447 ms 100.00% - 18 131.413 ms 100.00% - 19 134.001 ms 100.00% - 20 137.860 ms 100.00% - 21 135.767 ms 100.00% - 22 118.226 ms 100.00% - 23 116.347 ms 100.00% - 24 144.570 ms 99.93% 1m 2s 25 116.819 ms 100.00% - 26 114.652 ms 100.00% - 27 114.989 ms 100.00% - 28 131.390 ms 100.00% - 29 115.706 ms 100.00% - 30 126.691 ms 100.00% - Average Response Time: 138.076 ms Uptime %: 99.96% Downtime: 2m 0s Reponse Time Graph (Click for full size) [ATTACH]8[/ATTACH] Uptime Graph (Click for full size) [ATTACH]7[/ATTACH] Mars had a few 1 minute outages which I'm not sure about it could have just been us restarting the service. The other downtime that is not showing up is the few minute outage that happened when upgrading MySQL.

Mercury Uptime Report Date Average response time Uptime Downtime 1 N/A N/A N/A 2 N/A N/A N/A 3 N/A N/A N/A 4 N/A N/A N/A 5 N/A N/A N/A 6 N/A N/A N/A 7 N/A N/A N/A 8 N/A N/A N/A 9 N/A N/A N/A 10 N/A N/A N/A 11 N/A N/A N/A 12 N/A N/A N/A 13 N/A N/A N/A 14 111.736 ms 100.00% - 15 165.641 ms 100.00% - 16 165.509 ms 100.00% - 17 151.463 ms 100.00% - 18 131.236 ms 100.00% - 19 133.662 ms 100.00% - 20 157.114 ms 100.00% - 21 149.697 ms 100.00% - 22 153.238 ms 100.00% - 23 121.204 ms 100.00% - 24 144.042 ms 100.00% - 25 125.083 ms 100.00% - 26 111.021 ms 100.00% - 27 108.930 ms 100.00% - 28 128.004 ms 100.00% - 29 109.852 ms 100.00% - 30 112.301 ms 100.00% - Average Response Time: 134.102 ms Uptime %: 99.97% Downtime: 0s Reponse Time Graph (Click for full size) [ATTACH]6[/ATTACH] Uptime Graph (Click for full size) [ATTACH]5[/ATTACH] Great month for Mercury once we started monitoring it with the only down time being on the 31st when we did our MySQL 5 upgrade. The downtime for the MySQL upgrade was about 3 minutes.

Venus Uptime Report Date Average response time Uptime Downtime 1 N/A N/A N/A 2 N/A N/A N/A 3 N/A N/A N/A 4 N/A N/A N/A 5 N/A N/A N/A 6 N/A N/A N/A 7 N/A N/A N/A 8 N/A N/A N/A 9 N/A N/A N/A 10 N/A N/A N/A 11 N/A N/A N/A 12 N/A N/A N/A 13 N/A N/A N/A 14 129.605 ms 100.00% - 15 164.903 ms 100.00% - 16 174.917 ms 100.00% - 17 160.744 ms 100.00% - 18 123.839 ms 100.00% - 19 120.646 ms 100.00% - 20 132.949 ms 100.00% - 21 152.147 ms 100.00% - 22 134.911 ms 100.00% - 23 116.185 ms 100.00% - 24 146.031 ms 100.00% - 25 118.677 ms 100.00% - 26 113.742 ms 100.00% - 27 108.384 ms 100.00% - 28 161.348 ms 100.00% - 29 109.354 ms 100.00% - 30 113.868 ms 100.00% - Average Response Time: 134.250 ms Uptime %: 99.97% Down Time: 0s Reponse Time Graph (Click for full size) [ATTACH]4[/ATTACH] Uptime Graph (Click for full size) [ATTACH]3[/ATTACH] Overall good month for Venus there was about 3 minutes of downtime on the 31st not displayed which was the result of the MySQL 5 upgrade.

Here's Apollo's uptime report Date Average response time Uptime Downtime 1 N/A N/A N/A 2 N/A N/A N/A 3 N/A N/A N/A 4 N/A N/A N/A 5 N/A N/A N/A 6 N/A N/A N/A 7 N/A N/A N/A 8 N/A N/A N/A 9 N/A N/A N/A 10 N/A N/A N/A 11 N/A N/A N/A 12 N/A N/A N/A 13 N/A N/A N/A 14 122.337 ms 100.00% - 15 155.161 ms 100.00% - 16 200.903 ms 100.00% - 17 151.009 ms 100.00% - 18 118.386 ms 100.00% - 19 156.229 ms 100.00% - 20 257.491 ms 98.34% 23m 52s 21 139.707 ms 100.00% - 22 130.729 ms 100.00% - 23 113.946 ms 100.00% - 24 134.958 ms 100.00% - 25 118.054 ms 100.00% - 26 117.699 ms 100.00% - 27 115.959 ms 100.00% - 28 134.002 ms 100.00% - 29 111.943 ms 100.00% - 30 122.171 ms 100.00% - Average Response Time: 141.217 ms Uptime %: 99.87% Total Downtime: 23m 52s Reponse Time Graph (Click for full size) [ATTACH]2[/ATTACH] Uptime Graph (Click for full size) [ATTACH]1[/ATTACH] Ok first of all I have no idea why it does not display the 31st but it had about 5 minutes of down time when we upgraded to MySQL 5. On the 14th we had a large ddos which resulted in several outages of less than a minute to a few minutes at a time. The attack to mitigated and things went back to normal. Overall Apollo's uptime was quite good considering. This was only from the 14th on so it easily met 99.9% uptime for the month. This was actually one of the first times the machine has had any downtime more than several minutes for a month.

March was a shortened month as we only started doing the new monitoring on the 14th. So keep in mind this dataset is inaccurate and does not reflect the overall uptime that month actually had. I'm going to try to explain any downtime reflected on the reports as well as we generally know why something went down. This report also includes a few images of the uptime. Feel free to use them as you wish.

This has been completed with about 3-5 minutes of down time per server in order to make PHP compatible with MySQL 5

On Saturday March 31st between 1pm EST and 5pm EST we

Date: 3/22/2007 Start time (EST): 1:00am End time (EST): 5:00am Services affected: SoftLayer Verio Backbone Location: cer01.dal01 Duration: 4 hours Verio engineers on Thursday March 22nd will be performing maintenance on their Dallas network during the window stated above. Customers may experience service instability via SoftLayer