CPanel migration to Virtualmin - Email data does not copy over

26 posts / 0 new
Last post
#1 Wed, 04/29/2015 - 07:11
voxframe

CPanel migration to Virtualmin - Email data does not copy over

Hello,

We are trying to migrate from a CPanel host to Virtualmin.

We make a full backup, do the migration in Virtualmin, everything seems to go perfectly.

However the email accounts are all created properly, but empty. There is no actual email content in the accounts.

I have verified and there are accounts with data in them in the CPanel system, and I have verified that the email content is in the backup image.

After searching the forums, I see this seems to be a rather common problem, but all the threads I've seen haven't actually posted a solution, they just die off.

Is there a solution for this?

Thanks!

Wed, 04/29/2015 - 09:29
andreychek

Howdy,

I think you're seeing this bug here:

https://virtualmin.com/node/36922

Jamie says that will be corrected in the 4.17 release.

Since you and others have run into that, we're going to try and get a new release out soon in order to allow your migrations to complete successfully.

-Eric

Wed, 04/29/2015 - 09:39
voxframe

Thanks for the reply and info, greatly appreciated!

I'll keep my eyes open for 4.17. Any rough ETA on the release? (Days/Weeks?)

Thanks!

Wed, 04/29/2015 - 10:15
andreychek

Howdy,

I hope it's only "days" -- since you and others are unable to perform a migration, that's definitely not a good thing, and we want to get that new version pushed out soon. I'm going to talk to Joe and Jamie about getting that pushed out this week.

If for some reason that doesn't happen -- feel free to post a followup here, and maybe we can roll a custom version of Virtualmin just for fixing that issue.

-Eric

Wed, 04/29/2015 - 10:29
voxframe

Thanks again for the reply. (Great support!)

I'm in no immediate rush, so please take your time :-)

Fri, 05/15/2015 - 08:40
voxframe

Any update on this situation? I noticed the new 4.17 came out and people were still having the same issue.

Is there another release scheduled to come out for this issue?

Thanks!

Fri, 05/15/2015 - 10:33
andreychek

If you're still seeing this issue, would it be possible for the Virtualmin staff to see an example backup that's causing this problem?

If so, could you open up a new Support request, using the "Support" link above, and provide us a link to a backup containing this problem? You can mark the request as private to ensure only the Virtualmin staff can see that data.

Thanks!

-Eric

Fri, 05/15/2015 - 12:41
voxframe

Unfortunately I won't have the ability to send out the backup images.

But I notice that since the 4.16 and 4.17 update, the issue still remains exactly the same (And from what I've been reading, others are noticing the same issue.) So I'm assuming the fix didn't make its way into the stream yet.

I did a fresh test from today and still the same results. Manually opening the backup files reveals all the email data is there intact. So there is something somewhere in the migration scripts that aren't working right.

Fri, 05/15/2015 - 13:06
andreychek

Howdy,

Jamie did apply the fix to 4.17, but it either didn't work properly, or there's a second problem.

The issue is that we don't have a backup file of our own which we can use to reproduce that problem... we need a way to test and troubleshoot that.

If anyone else happens to have an example backup file we could use for testing, that would be helpful -- thanks!

-Eric

Mon, 06/29/2015 - 06:07
Peter Voskuilen

Hi,

When migrating from cpanel to virutalmin i have the same issues. the mail is not moved / migrated to the new mailbox.

If needed, I can provide a backup file, please contact me.

Mon, 06/29/2015 - 06:57
voxframe

That would be great if you guys could have a look at his backup file.

I can't get the green light to send the file over (Not my call). But I know by manually opening it up and looking, the mail is in there.

I didn't see a difference with 4.17 with this issue. Anyone else have this fixed for them in 4.17?

Wed, 07/01/2015 - 03:13
Peter Voskuilen

andreychek, can i send you an email with the location of a backup set that has issues with the mail?

Wed, 07/01/2015 - 10:17
andreychek

Howdy,

Yes, feel free to send me a link to the location of a backup file that exhibits this behavior. If you could, also include a link to this thread in the email body so I remember where to post once we sort out what the issue is :-)

You can email that to me at eric@openthought.net.

Thanks!

-Eric

Thu, 07/02/2015 - 16:04 (Reply to #13)
JamieCameron

Thanks for the backup file - I'm taking a look at it now. Can you let us know which users in which domains had email on cPanel though? The only user I'm seeing who has messages is the domain owner, and in my tests this does get migrated (Virtualmin 4.17, Webmin 1.760).

''

Thu, 07/02/2015 - 17:02 (Reply to #14)
andreychek

Actually Peter, since some of this might contain sensitive information to discuss here in the public forums -- would you like to create a private Support Request where we can go over some of those questions?

You're welcome to create a new Support Request using the Support link above... there, let us know that you were the one who sent us a cPanel backup, and let us know the names of the users that you expected to see migrated.

Thanks!

-Eric

Fri, 07/03/2015 - 02:45 (Reply to #15)
Peter Voskuilen

Eric,

I have created a new private forum post, since is was not able to find any other support option. Hope this will do.

Peter

Fri, 07/03/2015 - 08:26
voxframe

If possible, could you please keep this topic up to date with any progress/results?

I'm still sitting in the same situation and hoping your work will shed some light on the issue.

Thank you for all of your help. It's very appreciated!

Fri, 07/03/2015 - 08:33 (Reply to #17)
Peter Voskuilen

I'll keep you updated.

Depending on what comes from Eric and other support persons today or later this weekend I'll start a migration test on a brand new 4.17 virtualmin installation to see if this make any difference.

Wed, 07/29/2015 - 07:04 (Reply to #18)
Peter Voskuilen

Hi Voxframe,

Well, It took us some time, but in the end there is a solution, or well, atleast on my server.

Jamie from virtualmin/webmin did not got any errors on his platform with the migration/backup file I provided.

Tried some suggestions / patches and so on without any luck.

So I gave Jamie access to my server to try to solve the issue.

Jamie managed to discover that is was some kind of bug in a Webmin-provided API. this bug will be corrected in the next release.

Sat, 07/04/2015 - 02:29
rubenz

I got the same problem. Solved by manually copying mail files (they were kept in Maildir format).

Mon, 07/06/2015 - 10:37
andreychek

We are looking into this issue -- we hope to be able to include a fix for this in the next Virtualmin version.

-Eric

Wed, 07/29/2015 - 17:23
andreychek

As Peter Voskuilen mentioned above, Jamie did discover the source of the problem.

Once he discovered the problem, it actually turned out to be a simple fix.

It's just a matter of editing the file mailboxes/boxes-lib.pl, and in there, jump down to line #2961 where you'll see this line:

my $rv = rename($oldfile, $newfile);

That line needs to be changed to look like this:

my $rv = &rename_file($oldfile, $newfile);

That change will be included in Virtualmin 4.19.

-Eric

Wed, 09/09/2015 - 08:48
voxframe

Is there any update or ETA for this fix? It's now starting to get critical that we get this operational.

I tried to do the above patch, but cannot locate these lines in any of the boxes-lib.pl files.

Which boxes-lib.pl file am I supposed to edit? (Exact location) And what is the updated line number? I don't even find a match for those above lines anywhere in the file.

Thanks!

Tue, 11/03/2015 - 12:38
voxframe

Has there been any update for this?

We need this functionality urgently and are willing to pay development.

We need to import CPANEL backups including mainly user email/settings.

At my last knowledge, the latest updates still do not fix this issue. Please advise if anyone knows if this is fixed for certain.

Tue, 11/03/2015 - 13:21
andreychek

Howdy,

Which Webmin and Virtualmin version is it that you're using there?

It's possible that the line number in question is different in the version you have.

That file appears to be within Webmin, rather than what I mentioned above, Virtualmin -- you may just need to do a search on the string in question within Webmin's mailboxes/boxes-lib.pl folder.

For me, it's on line 2926 now, within the "rename_as_mail_user" function.

-Eric

Tue, 11/03/2015 - 13:30
voxframe

Virtualmin 4.18 Webmin 1.760

Please give full pathway (Or partial... anything) to this file?