Backup on Google Storage (bucket & path)

18 posts / 0 new
Last post
#1 Wed, 12/31/2014 - 06:14
guy.froissart@g...

Backup on Google Storage (bucket & path)

Hi What do I have to put in fields " Bucket and Path "? Look at the image as attachment. I do not find this information about my account " Google Storage "

Generally I use AMAZON S3 this information (bucket & path) is well identified for AMAZON

Wed, 12/31/2014 - 06:39
PaliGap

I can't get Google Storage to work either ;-)

In the "Cloud Storage Providers" section I have entered my project name as per the Google Storage browser.

At Google I have created a bucket and a folder. In the Google Storage browser this is displayed sometimes as "bucket/folder/" and sometimes as "bucket/folder". If I enter the first option (trailing slash) I get an error:

"Failed to save scheduled backup : Missing Google Cloud Storage page". In fact I find that all these formats fail:

"/bucket/folder"
"bucket/folder/"
"/bucket/folder/"

If I enter in "bucket/folder" format, that is accepted OK. So I think that must be it. But that still doesn't work for me. When I try to run it I get

"Starting backup of 1 domains to test in Google folder [bucket]..

Invalid argument."

Wed, 12/31/2014 - 06:46
PaliGap

Ah! Some progress!

In the Google Storage browser I see that the Project has a common name and a "proper" name called "Project ID". So I went back to the Cloud Storage Providers" section in Virtualmin and entered that instead. Now, when I run the backup something starts to happen! But... :-(, now I get a different issue:

"Creating final backup archive ..
.. done

Uploading archive to Google Cloud Storage ..
.. upload failed! Invalid Value

Backup failed! See the progress output above for the reason why."

Wed, 12/31/2014 - 06:57 (Reply to #3)
guy.froissart@g...

I don't see this

Is it in Stockage ==> Google Storage ==> Browser ? or ==> DashBoard Project ?

I saw well Project ID but it is the number which is posted(shown)

Wed, 12/31/2014 - 07:16
PaliGap

In the Google Developer's Console, the Project ID is shown next to the project name when you select "Projects".

Click on the project and then "storage browser" to see the buckets and paths.

Interestingly there is a THIRD candidate for referrring to the project when you select the project >> project dashboard. That's

"Identifying your project
When creating new buckets, or listing the buckets owned by a project, you must specify the x-goog-project-id HTTP header in order to identify which project you are using.
x-goog-project-id: xxxxxxxxxxxxx"

Oh Lordie!

And what about another option shown there?

"Interoperable Access
Interoperable API access lets you use HMAC authentication and lets Google Cloud Storage interoperate with some tools written for other cloud storage systems. If you don't need this ability, don't enable interoperable access. You can enable this later if you discover that you need it. Clicking this button will enable interoperable API access, and will use this project for all create bucket and list bucket requests when using the interoperable API."

Perhaps I'll try hitting that button!

Wed, 12/31/2014 - 07:32 (Reply to #5)
guy.froissart@g...

put you : " x-goog-project-id " in bucket & path ?

Or only the number ?

I put it under the shape: " x-goog-project-id: xxxxxxxxxxxxx " and it does not run

After I put in the field bucket & path the number : "xxxxxxxxxxxx" then I have this result "Refresh failed : HTTP/1.0 401 Unauthorized" ==> Backup failed

Wed, 12/31/2014 - 07:31
PaliGap

Success!

I did NOT need to mess about with the "Interoperable Access" option. I had in fact fixed it already by using the project ID (and not the project label - see above). The reason it did not work immediately is that I had been playing with the bucket name. When I put that back to "bucket/folder" format it worked perfectly.

(By the way Guy, I'm testing Google out as I have found the Amazon S3 backup to be a bit unreliable. What's your experience with it?)

Wed, 12/31/2014 - 07:46 (Reply to #7)
guy.froissart@g...

Is the project id is bucket or folder ?

Is the form is project_id/folder ?

Now what you put for folder ?

Wed, 12/31/2014 - 07:38
guy.froissart@g...

Personally I prefer S3 it is much easier for the implementation. I have never had a problem with AMAZON S3 But more expensive for a long use.

Wed, 12/31/2014 - 08:54
PaliGap

To sum up:

1) In the Google Developer's Console, the Project ID is shown next to the project name when you select "Projects".Go to Cloud Storage Providers. Enter that at Virtualmin Cloud Providers.

2) At Google, click on the project and then "storage browser" to see the buckets and paths. Enter that at Virtualmin in the backup schedule in the format "bucket/path".

Wed, 12/31/2014 - 09:14
guy.froissart@g...

I try but it does not work I have always this message "Refresh failed : HTTP/1.0 401 Unauthorized"

When you make the OAuth Client ID : did you use well in "applications type" item 3 : Installed Applications and not "Web Apllications" or "Service Account" ?

Wed, 12/31/2014 - 09:59
PaliGap

I used "installed application " >> "other"

To sum up (as far as I can remember!):

To generate authorisation, I used this guide:
https://www.virtualmin.com/documentation/cloudmin/vm/gce-account

At Google I created a project. I gave it a name, but Google generates an ID. This is confusing as when Virtualmin requests the project name, it really needs the project ID! I also created a “bucket” and a folder

In VM Cloud providers you can then enter your email address, your OAuth2 client ID, your OAuth2 client secret and your project “name” ( (i.e. ID!!!).

Finally when creating a backup schedule you enter the destination as “bucket/folder” (no leading or trailing slash).

Wed, 12/31/2014 - 10:40
guy.froissart@g...

It is exactly what I realized nevertheless I always have the same result :

this is Virtualmin Message :

Starting backup of 1 domains to save in Google folder backupsrv122 ..

Refresh failed : HTTP/1.0 401 Unauthorized

Wed, 12/31/2014 - 11:00 (Reply to #13)
guy.froissart@g...

Nevertheless I tried Project ID (xxxxxxxxxxx) and Project Name (abcdefghijk) in Virtualmin ==> name of the project Nothing works

Wed, 12/31/2014 - 11:03
PaliGap

Well I'm no expert Guy! ;-)

You seem to have something wrong with the credentails/permissions there. Here'a an idea:

You have entered your Google project ID at Virtualmin? Now at Google, select your project and then select "permissions". When I do that I see two entries. The first one is [project-id]@appspot.gserviceaccount.com. Do you have that too?

Below that it says "App Engine service account". You too? And on the right it says "can edit". You too?

Next, let's check "APIS & Auth". I see "OAuth" and beside it "Client ID for native application" - and there I have the client ID & secret that I have entered at Virtualmin. You too?

Wed, 12/31/2014 - 11:26 (Reply to #15)
guy.froissart@g...

It is really a 12-digit number for Project Name You add nothing else ?

Wed, 12/31/2014 - 11:22
guy.froissart@g...

1/ yes it's the same for : - [project-id]@appspot.gserviceaccount.com Authorized modification - [bucket]@appspot.gserviceaccount.com Authorized modification

2/ yes identical

3/ yes identical I even tried to change the secret key in this interface then in Virtualmin Make a new generation of key OAuth in virtualmin when we make save

Nothing works I always have this problem of authentification

Wed, 12/31/2014 - 11:37
PaliGap

My project ID is *not* a 12 digit number. It's alpha-numeric. Go to project >> overview. Are using the 12 digit "Prject number"? You need to use the project ID!

Also, at "[project-id]@appspot.gserviceaccount.com" I do *not* see "Authorized modification". I see "App Engine service account" and "can edit"

I think you may have generated credentials for the wrong type of service? I think this is correct:

1) Go to the console at https://console.developers.google.com/ , click on your project, and then on APIs & auth -> Credentials from the left menu.
2) Click on Create new Client ID
3) Select Installed application and Other , then click Create Client ID