Problem with backup to S3

21 posts / 0 new
Last post
#1 Mon, 04/13/2009 - 03:36
Anonymous

Problem with backup to S3

Hi,

I have opened a ticket about this 3 days ago but had no response so I am trying here. ( I went to support tickets in my VirtualMin menu )

I opened an amazon account and have a working "bucket". I have tested it by uploading a file using Firefox S3 organizer plugin. So the bucket does work.

Here is what I did to try and do a backup:

[img size=500]http://www.expert-world.com/im/test/backup.jpg[/img]

And this is the error message I get.

Starting backup of 31 domains to Amazon S3 bucket heavyhoster .. Failed to list S3 buckets : The difference between the request time and the current time is too large. Backup failed! See the progress output above for the reason why.

Any ideas how to solve this ?

I want to offer hosting but part of that is having a very reliable backup and S3 seems the best way to go.

Thanks PS - I run VirtualMin Pro 3.67 on CentOS Linux 5<br><br>Post edited by: Davvit, at: 2009/04/13 03:36

Mon, 04/13/2009 - 04:29
andreychek

Howdy,

I'm unfortunately not familiar with what would cause that (though I might verify your system time just in case).

However, this was a holiday weekend in the US, so those who may normally have responded to your Bug report may not have had as much time as usual to look at it.

I'd recommend posting a followup comment on your ticket, to remind folks to take a look at it :-)
-Eric

Wed, 04/15/2009 - 00:12
Davvit

Thanks for your reply.

I don'!t think I can comment on my ticket because I marked it private
and I can not see it in the task list.

Anyway I would like to check and correct my server time.

I have never done that - where can I do that from ?
Is there a virtualmin module that allows me to do that ?

- One other difficulty is that I am in Turkey - so I don't know what the &quot;correct&quot; time should be!
My server is in USA leased from: Rackmounted.com
Datarealm Internet Services, Inc. Not sure which state they are in but Phone number code is 602. (Full number: 602-850-4044 )

Should I go by the times displayed on these posts ?

Hope you can help me get this sorted- thanks.

Wed, 04/15/2009 - 00:17 (Reply to #3)
Joe
Joe's picture

<div class='quote'>I don'!t think I can comment on my ticket because I marked it private
and I can not see it in the task list.</div>

Private tickets should still be visible to the owner of the ticket. But, if it has been closed, it will no longer be visible in the default search. You can, of course, choose to look at closed tickets. Just open out the Advanced search (click it), and then choose &quot;All Statuses&quot; in the Status field, and enter your username in the Opened By field.

--

Check out the forum guidelines!

Wed, 04/15/2009 - 00:24 (Reply to #4)
Joe
Joe's picture

And yes, there is a module for time in Webmin. Webmin-&gt;Hardware-&gt;System Time.

It doesn't matter what time zone you choose for your server, but the time needs to be accurate for that time zone. I happen to set our servers time zone to the one where the servers live (Dallas and Houston, in the Central time zone). An area code of 602 puts your server in Phoenix, Arizona (might be reseller of servers hosted by GoDaddy who are the largest host in that area), which is Mountain time zone (UTC-7), and the timezone selector in Webmin has a Phoenix, Arizona setting.

Once you've got the time set correctly, you'll want to enable synching to a time server. Webmin can setup a cronjob for you, or you can just start the ntpd service (ntp is the network time protocol, which provides incredibly accurate time keeping). I usually use ntpd, but either is good enough for web service.

--

Check out the forum guidelines!

Sun, 06/07/2009 - 07:51
Davvit

OK - thanks for the info.

I have changed my system time Phoenix Arizona time zone,
and I then set hardware time to this new system time.

Hopefully thats the right thing to do.

Finally on the &quot;Time server sync&quot; tab, I have set it to an hourly
sync but it looks like it wants a url in this box:
<b>Timeserver hostnames or addresses:</b>

Is that where I put the &quot;ntpd&quot; that you mentioned ?

I googled and found the relevant website and it gives this info:

North America &acirc;

Wed, 04/15/2009 - 08:54
Davvit

Shall I put in &quot;north-america.pool.ntp.org&quot; ?

Will it know that I want to sync to Arizona tine zone ?

Well - I am trying it now :)

Wed, 04/15/2009 - 08:56
Davvit

OOPS

I got this error:

<div class='quote'>Failed to connect to north-america.pool.ntp.org:37 : Connection timed out</div>

Now I am stuck again :o

Sun, 06/07/2009 - 07:51
Davvit

OK - thanks for the info.

I have changed my system time Phoenix Arizona time zone,
and I then set hardware time to this new system time.

Hopefully thats the right thing to do.

Finally on the &quot;Time server sync&quot; tab, I have set it to an hourly
sync but it looks like it wants a url in this box:
<b>Timeserver hostnames or addresses:</b>

Is that where I put the &quot;ntpd&quot; that you mentioned ?

I googled and found the relevant website and it gives this info:

North America &acirc;

Wed, 04/15/2009 - 10:47
Joe
Joe's picture

As I mentioned, there are two possibilities. The information you found is for ntpd. Webmin does not use ntpd...it merely does a sync periodically via a cronjob. The latter is lighter weight (no server running all the time) but less precise. You can use one or the other.

<div class='quote'>Shall I put in &quot;north-america.pool.ntp.org&quot; ?</div>

Looks to me like the hosts are named:

0.north-america.pool.ntp.org
1.north-america.pool.ntp.org

...and so on. So, you'd want one of those names.

<div class='quote'>I guess that the Webmin module is updating this file for me</div>

No. Webmin just sets up a cronjob that polls the server periodically. It does not setup ntpd. Which is why I said, &quot;Webmin can setup a cronjob for you, or you can just start the ntpd service (ntp is the network time protocol, which provides incredibly accurate time keeping). I usually use ntpd, but either is good enough for web service.&quot;

ntpd is almost certainly already configured reasonably RHEL/CentOS packages are configured, by default, to sync to the Red Hat time server, I think. You just need to start it, if that's how you want to sync time. Don't bother messing with the configuration.

--

Check out the forum guidelines!

Wed, 04/15/2009 - 21:10
Davvit

OK - thanks for clarifying it.

Sounds like I should do the ntpd.

Well I just checked the directory

var/lib/ntp and it doesn't exist :(

Under var/lib/ I have :
...
multipath
mysql
nfs
pgsql
...

no /var/lib/ntp/ !!

I also used putty and entered:
ntp
ntpd
ntpq -pn

They all came up with: -bash no such command.

So I guess that my system is missing ntp ?

BTW:

I also tried to enter :
1.north-america.pool.ntp.org

into that cron job box and got this back:
&quot;Failed to connect to 1.north-america.pool.ntp.org:37 : Connection refused&quot;

Wed, 04/15/2009 - 21:26 (Reply to #11)
Joe
Joe's picture

So install it! With yum that'd just be:

yum install ntp

--

Check out the forum guidelines!

Wed, 04/15/2009 - 23:44
Davvit

OK

did that :)

Checked directories and now there is a
var/lib/ntp

so thats good - but there is nothing in the ntp directory !

I wonder where that config file is ?

Also what do I need to do to start ntp ?

do I type &quot;run ntp&quot; ?

Thanks for your help, hopefully all this will lead to
getting that back up working :)

Thu, 04/16/2009 - 00:27 (Reply to #13)
Joe
Joe's picture

Config files do not go in /var/lib.

Config files go in /etc.

But you don't need to configure it. The stock package, I'm pretty sure, is already configured to use Red Hat's time servers.

Services on Red Hat based systems are started with the service command:

service ntpd start

Or you can use the Webmin Bootup and Shutdown module to start and stop services.

You'll also need to set it to start on boot, which is done with chkconfig.

chkconfig --level 345 ntpd on

Again, this can be done in the Webmin Bootup and Shutdown module.

--

Check out the forum guidelines!

Thu, 04/16/2009 - 08:57
Davvit

Thanks for your help,
I now have my hardware and system times in sync on Arizona tine
zone and synced with ntp.

Unfortunately, I just tried the back up again
and I get the same error message:
<div class='quote'>
Starting backup of 31 domains to Amazon S3 bucket heavyhoster ..

Failed to list S3 buckets : The difference between the request time and the current time is too large.

Backup failed! See the progress output above for the reason why.
</div>

Does anyone else use S3 to back up ?
Is it working OK ?

This is frustrating, I have my &quot;bucket&quot; sitting there and I can not
use it :(

Would Jamie know about this module and why I am getting this error ?

Thu, 04/16/2009 - 10:13 (Reply to #15)
andreychek

So if you log into your machine using SSH, and type &quot;date&quot;, what output do you get?
-Eric

Thu, 04/16/2009 - 13:07 (Reply to #16)
Joe
Joe's picture

Lots of people are backing up to S3 (including us for some things). I think this error is legit. I'm guessing you still don't have accurate time on your server. Check /var/log/messages for ntp related warnings...it will die if the time is too far off when it starts (changing time rapidly is dangerous because it can cause mail-related issues; Dovecot will shut down if you change time dramatically for example).

--

Check out the forum guidelines!

Mon, 04/20/2009 - 03:46
Davvit

At the time of writing and editing this post I
have just run the date command and I get this.

[root@heavyhoster ~]# date
Mon Apr 20 05:45:11 MST 2009
[root@heavyhoster ~]#

Mon, 04/20/2009 - 03:50
Davvit

I just logged into my server and it says:

<div class='quote'>Time on server 20/Apr/2009 05:48</div>

Your forum server seems to be 2 hours different from my
Pheonix, Arazona based server. Is that about correct ?

Thanks.

Mon, 04/20/2009 - 04:05
Davvit

I just looked at my mail log and the last entries are:

Apr 20 05:09:33 heavyhoster postfix/qmgr[23170]: 03D567AE265: removed
Apr 20 05:12:40 heavyhoster postfix/anvil[32519]: statistics: max connection rate 1/60s for (smtp:88.226.251.95) at Apr 20 05:09:19
Apr 20 05:12:40 heavyhoster postfix/anvil[32519]: statistics: max connection count 1 for (smtp:88.226.251.95) at Apr 20 05:09:19
Apr 20 05:12:40 heavyhoster postfix/anvil[32519]: statistics: max cache size 1 at Apr 20 05:09:19

I also tried to access my email via Outlook Express and none of my accounts are respondeing - I get this error:

Your server has unexpectedly terminated the connection. Possible causes for this include server problems, network problems, or a long period of inactivity. Account: 'aa-ads.com', Server: 'mail.aa-ads.com', Protocol: POP3, Port: 110, Secure(SSL): No, Error Number: 0x800CCC0F

and:

The host 'mail.fethiye-guide.com' could not be found. Please verify that you have entered the server name correctly. Account: 'Fethiye-guide.com', Server: 'mail.fethiye-guide.com', Protocol: POP3, Port: 110, Secure(SSL): No, Socket Error: 11004, Error Number: 0x800CCC0D

I will try and close down and restart the Postfix server - may be that will help ?

Mon, 04/20/2009 - 04:53
Davvit

SUCCESS !!

When I went to re-start the mail server I noticed that Dovecot was down
so I re-started both postfix and Dovecot and now both my Outlook Express is accessing the server and the S3 backup worked !

So,

Thanks for your help :)