CF Database corrupted again on 27 January with 28 hour downtime, now fixed!

[Erwin]

As members would know, CF had a database crash on 25 January caused by the wrong setting in a config file - I fixed it in about 12 hours with no data loss.

Well, I didn't learn my lesson and kept modifying the database config to try to optimize the database so I could put our "find member's posts" feature back in.
It worked but because of the way I recoded things, the database was cacheing a lot of disk writes into RAM, and this caused occasional writes not to be written - leading to another database corruption.

This database corruption affected the WHOLE database - I had to restore our backups 3 times but they ended up being corrupted, and in the end decided to try to fix up the corrupted database. Took about 12 ours again...

Praise God, I fixed it, with minimal or no data loss whatsoever.

So now CF is back.

I promise I won't try to be too smart now. I'll leave things alone!

 

[AngelAmidala]

All hail you Erwin!

 

[MKalashnikov]

Good Job Erwin.

 

[Deamiter]

Horray for fixing a 24 hour problem! Just out of curiosity, IS the search feature back, or have you learned your lesson there too?

 

[Marie]

Woo hoo!

 

[Erwin]

Horray for fixing a 24 hour problem! Just out of curiosity, IS the search feature back, or have you learned your lesson there too?

I couldn't make it work without the database going crazy.

I'm now at work, so I'm going to have to concentrate. I woke up at 4 am this morning to work on CF... it's been a crazy CF week for me.

 

[sinner/SAVED]

You are amazing, Erwin!

 

[Plan 9]

Erwin, thanks for trying to restore the "find members' posts" feature; I really miss it.

 

[Iron Sun 254]

Thanks for all the hard work you do to keep this place running. I'm lucky that I never worry about being too smart.

 

[tjboie2001]

Thank you Erwin for fixing CF

 

[ukok]

Erwin, we already know you are smart, you don't need to play around doing fancy stuff to prove it

Well done!

God Bless.

 

[Revelations 22:20]

Go Erwin

 

[brother daniel]

Thank you Erwin God is with you

 

[CPman2004]

Thanks Erwin for all the hard work. I guess the mini vacation for staff is done now

 

[brinny]

Praise God!!! I was just praying before I logged in. God bless you Erwin!!!

 

[TheBear]

As members would know, CF had a database crash on 25 January caused by the wrong setting in a config file - I fixed it in about 12 hours with no data loss.

Well, I didn't learn my lesson and kept modifying the database config to try to optimize the database so I could put our "find member's posts" feature back in.
It worked but because of the way I recoded things, the database was cacheing a lot of disk writes into RAM, and this caused occasional writes not to be written - leading to another database corruption.

This database corruption affected the WHOLE database - I had to restore our backups 3 times but they ended up being corrupted, and in the end decided to try to fix up the corrupted database. Took about 12 ours again...

Praise God, I fixed it, with minimal or no data loss whatsoever.

So now CF is back.

I promise I won't try to be too smart now. I'll leave things alone!

Thanks, Erwin!

 

[rachewil15]

Muchas Gracias Erwin!!

 

[Sephania]

I was praying for you Erwin as I am sure you were tearing your hair out trying to do this and have work and everything.

Just a quick question, I noticed when it first went haywire that while you could get into the opening page and chat but not the individual forums, you could access the PM and send and receive. Now I don't know a whole lot about databases, but this would seem to me to be on a different one? The PM's I mean. So if that is true, couldn't you somehow split the forums so as the "lighter" forums would be on a data base where you can't search for posts and the more "in-depth" forums would be on another that would have the search capability?

Just thinking, it really is a lose but I dont' think a necessity for the whole of CF.

Glad you can rest now, I don't know how you do it, the L-RD has blessed you mightely!

 

[Palatka44]

Erwin,
I thought that I had caused CF to crash as I had just changed my Signature when every thing went down. Whew! I am not the cause, the burdens been lifted and now CF is back!
May you always have more smarts then me.

 

[OrthodoxyUSA]

Erwin,

I cant sometimes can't imagine why you would want to keep up with this.....

God Bless You...

Forgive me..