How to notify a person when this goes down?
I made a simple notifier command line application in C#. Basically I use the windows task scheduler and set it up to run this command line application at a certain time.
When this time is hit the command line application is ran all notifications are grabbed from a database(using linq to sql) formatted and sent by smtp to the right people.
Now how should I handle these scenarios
- A database error occurs
Solution: Send a email to the admin to notify them that it failed and to check it out.
- A smtp error occurs
Solution: ?????
So these are really the two things that could go wrong. There could be di开发者_Go百科fferent combination's of this.
Database error might occur and and smtp might occur too, Or only one or the other might occur.
So how to get this information to an admin or someone so they can fix it. I highly doubt that an admin would go and check every single day to see if the notification thing worked or not.
So basically how to make it fail safe or at least make it so that if something goes wrong a admin can come fix and it and just run the notifier manually and get everything back in sync.
You have a couple options.
Write the error information to a log. Make it a procedure for some operator to check the log every so often.
Have it call some other service?
However, if the database is used by more than just your app, it's entirely possible that someone else will notice. Same thing with the email server. If it's down, then I bet a lot of people will be making phone calls. Note that your situation precludes lost network connectivity (like a dead switch or misconfigured router).
Incidentally if it detects SMTP is back up, then it should notify someone that it was down for a given time period. Same thing about the database server.
Finally, ALL error conditions should be written to your system log. It's pretty trivial to write to the windows event logs. If your system administrators are following commonly accepted protocols they should be monitoring those pretty often anyway.
Logging an Error to the EventLog could also be an option, providing that the Admin either checks the EventLog regularly or that you have monitoring systems that aggregate those errors.
You could use an SMS gateway or attach a cellphone to the server and have it send a text message to an admin when something has gone horribly wrong.
But you can take this even further:
- What if you lose network connectivity (a switch or cable gone bad for example)?
- What if the power goes down (when is the last time you tested the UPS)?
Depending on your needs (and the business cost of not sending out these notifications in short notice), you may need a full-fledged monitoring solution.
So the main problem is that you do not know how to notify the admin in case of an SMTP server failure. You have several options, assuming the admin's e-mail account is not residing on the same server.
First option: You could set up a cgi mail script on another server and contact that via http to send the e-mail.
Second option: Use an smtp client and set it up to contact the receiver's smtp server for sending the e-mail directly (instead of going through the local smtp server).
BTW: Some e-mail providers offer text message notifications for e-mails originating from a certain address. That's very handy if the server goes down while the admin is away from his desk.
For your DB errors, you could hardcode in a (set of) email address for DB errors to be sent to.
As for the SMTP errors, you can either do a "send email when smtp recovers", or you have to have an alternate method for alerting your admins. This could be a backup SMTP server, a SMS gateway (as someone else mentioned), or even something such as someones desktop in the network popping up an alert if it doesn't receive some type of "everything is ok" alert/message from the box running the script.
精彩评论