Date: Thu, 23 Nov 2000 15:32:17 -0500 (EST) From: BCPL.NET SysAdmin To: BCPL.NET News Subject: BCPL.NET NEWS: Unplanned Down Time ------------------- UNPLANNED DOWN TIME ------------------- In the late afternoon and early evening of Wednesday, November 22, the mail queue on our mail server (MAIL.BCPL.NET) became supersaturated with messages waiting for delivery. The load of trying to process this huge mail queue slowed the mail system to a crawl. Contributing to the slowdown, during that period an unusually large number of customers were trying to send and receive e-mail, and new mail was arriving from outside at a tremendous rate. The result was that anyone who tried to do anything with e-mail e-mail or in a UNIX shell account during that time period experienced serious performance problems. By about 7:30 PM MAIL.BCPL.NET was bogged down so badly that I couldn't do anything more to fix the problem. It was like trying to walk through a vat of molasses. Drastic measures were required to reduce the load on the server. I shut down the modem pool to prevent customer access to the mail system (and other services on MAIL.BCPL.NET). I also reconfigured our gateway router to prevent access from outside BCPL.NET. Cleaning up the mess in the mail queue and the hundreds of "POP Lock" files left behind by timed-out POP mail sessions took a lot longer than I expected. I did what I could to salvage everything in the queue, but it is possible that some mail was lost. Finally at 10:00 PM MAIL.BCPL.NET was back to normal. I brought the modem pool back on line and opened the gateway router to the rest of the world. The modems filled up very quickly and the mail flow from outside was truly astonishing, but MAIL.BCPL.NET didn't miss a beat. Phew! Sorry about the interruption in service. Have a good Thanksgiving! Chip -- BCPL.NET INTERNET SERVICES CONTACTS: ----------------------------------- Web Site: http://www.bcpl.net Administration & Policy: ispadmin@bcpl.net 410-887-6180 Sales, Renewals, Account Status: accounts@bcpl.net 410-887-4172 Technical Support (Help Desk): help@bcpl.net 410-887-3297 Usenet News Newsgroup Requests news-admin@bcpl.net 410-887-6180 E-Mail & Newsgroup Abuse Reports: abuse@bcpl.net 410-887-6180 Domain Name Service Issues: dnsadmin@bcpl.net 410-887-6180 FAX: 410-887-2091