Synchronize two folders on Android with all subfolders and files in them?

Been brushing up on some knowledge in the past couple weeks and have something that might work. I am glad to see that rsync is mentioned already in here. I believe an ssh connection between your PC and android would allow you to use rsync and accomplish what you would like. I would try it myself and give you a definative answer but I am still not sure that I completely understand what you are EXACTLY trying to do. What I mean is, that I think I know what you are going for, but if I wouldnt like to work on the quirks of a particular issue only to find that I had the wrong idea (Im sure there are those in the forums here that can sympathize)

Alternatively I see no reason why you could not simply mount your device as a removable drive in a linux system and then use rsync (if ssh will not let you use rsync as I think).

I absolutely must be missing something here but from where I sit right now I see no reason that you wouldnt be able to do this with the above mentioned rsync https://en.wikipedia.org/wiki/Rsync I really hope this is solved for you now.

Thank you for posting. We have a situation where I see the issue being 'beyond strange' and others do not. I would like to clarify the topic so you can understand... and if you can help me, then you would be the first person to do so ever since it became impossible to mount an SD card as a drive letter on Android.
So here we go with the clarifications:

Topic is this: modern Android cellphones no longer have a USB Mass Storage Mode
Modern Android cellphones only have MTP (Media Transfer Protocol) or PTP (Picture Transfer Protocol)...

Since there is no USB Mass Storage Mode, you cannot mount your SD card as a drive letter. Now if you could mount your SD card as a drive letter - problem solved as I could connect the phone to PC and sync that drive letter in SECONDS. However, since you cannot mount the SD card as a drive letter, syncing is a THOUSAND times slower.


The person above you posted how it would suck the "first time having it wait" and implied how thereafter you wouldn't have to wait.... but this is entirely incorrect.


I have a 10 GB folder on my PC filled with text files, links, programs, all in all fifty thousand files in five thousand folders.
Let's say an identical folder exists on an Android phone.


Every time even a tiny txt file is changed on my PC it-would-take-FOREVER to compare the folder on PC with folder on the phone using ANY method to discover that a single tiny txt file needs to be updated on the phone.

This can be tested with a stop watch.



Example two.
If I hold in my hand a microSD card with a 10 GB folder (with many subfolders) and I want to sync that with both PC and phone.
[Let's say identical folders exist on a PC and an Android Cell Phone, to make things easier]


I insert that microSD card into my PC/Laptop and I can sync it a thousand times quicker than I could if I inserted the same microSD card into a cell phone and tried to sync an identical folder on the cell phone.



As for direct folder syncing on Android: I talked to developers on xda... there is no way to quickly do what I just described, none, it doesn't exist. That to me is beyond strange - here we are almost in 2016 and you cannot directly sync two folders on Android.

This is an old thread on which I chose to end the discussion earlier because of the 'why would you want to do that, no one wants to do that' approach that people have to this problem

c627627 said:

This is an old thread on which I chose to end the discussion earlier because of the 'why would you want to do that, no one wants to do that' approach that people have to this problem

Click to expand...

Sorry didnt mean to revive an old thread. But I think this is odd as well, and may need it in hte future, so I revisit it every once in a while.

you cannot mount your SD card as a drive letter. Now if you could mount your SD card as a drive letter - problem solved as I could connect the phone to PC and sync that drive letter in SECONDS

Click to expand...

Sooooo... Ive got good news and bad news on this front.... Good news is: It can be done. Mounted to a drive letter that is. http://android.stackexchange.com/questions/22979/drive-letter-for-mtp-connection-under-windows This does work. I tested it myself. While I normally would just drop the link in your lap and say "GO see what you can find" I have tested this -because I may need to sync someday. Bad news is that you must be able to use WiFi on your local network, and you are limited to the speed of wifi, which IIRC is still mayn times better than USB3/3.1 but not as good as a hardwired CAT5/5e/6 connection. There is an addendum to the bad news as well... Ive only partially tested this solution. Ive got no syncing software and no need/want/time to search at the moment - this is the part where I get lazy and tell you to have at it- so I have no idea if this solution will function correctly within the software you choose. But it IS an sd card mounted with a drive letter, in windows no less (I was certain I would have to use Linux for this part)

I am still researching rsync and if it is viable for this type of purpose but have run into hardware stumbling blocks that I will need to work through - not to mention my limited understanding of the syntax associated with rsync - but as I said, I am still plugging away in that direction. I really do hope that your software will sync your card with your pc with this new drive letter

As you stated earlier that you are able to root your device and comfortable with those types of procedures I thought I would mention as well an app that I saw claiming it would allow "you to go to your sdcard in your phone and PC simultaneously". I wasnt sure if it would be of any interest to you , but it seemed as though it might be. I forget the name but it was found in a search for "mount as USB"

Good Luck!

knoober said:

you are limited to the speed of wifi, which IIRC is still mayn times better than USB3/3.1

Click to expand...

1. I've yet to see any phone with a USB 3 port. A micro-USB 3 port is physically different from a micro USB 2 port.
2. USB 3 is 5Gbps. Even with the overhead inherent in USB, that's faster than your 802.11ac network will ever be.


Regarding syncing as a drive letter and depending on a PC to do it, just install something like Servers Ultimate "Pack B" and export the SD card as an NFS or SMB share. No need for messing with MTP's crappiness. If you'd rather use rsync, that's in "Pack A".

Servers Ultimate looks promising. Thanks for the heads up

As far as the USB3 beating my wifi... no doubt. In my head I was seeing the 20-30Mb/s as Gb/s and though " yeah that's faster than USB3 OR 3.1... waay faster " My day just wouldnt be complete without that kind of mistake. I can only guess how many times I would have repeated that if you hadnt caught it!

knoober said:

Sorry didn't mean to revive an old thread. But I think this is odd as well, and may need it in the future, so I revisit it every once in a while.

Click to expand...

On the contrary, thank you.
I am now cautiously optimistic that I am not crazy thinking it's mind boggling that Windows users have a need for this to the tune of so many programs being available all over the place for Windows yet NO ONE ever anywhere thought there is a need for this on Android and we are looking at 2016 (!!) External and internal microSD cards CANNOT be synced directly!

The drive mounting yes can be done on certain phones, I have a program for Samsung Galaxy 4 but not for later modern models. I will look into your link and see if it works on my Samsung Galaxy 5. EDIT: wait, what did you use exactly to mount which phone to show up as a drive letter on PC?


Of course Wi-Fi is the problem here, WiFi is the limiting factor.

1. It has to be a cable connection through a mounted letter.
separate from that, separate issue:
2. Direct folder to folder sync on Android itself, external to internal microSD card sync.


I am absolutely comfortable with rooting and everything else, but those apps simply do not work on modern phones



petteyg359, the PC sync program NEEDS a drive letter to sync the connected phone a thousand times quicker than using any other method.
Perhaps I need to understand what you posted better but syncing PC to phone (superfast) requires a cable connection and a drive letter...

c627627 said:

petteyg359, the PC sync program NEEDS a drive letter to sync the connected phone a thousand times quicker than using any other method.
Perhaps I need to understand what you posted better but syncing PC to phone (superfast) requires a cable connection and a drive letter...

Click to expand...

Enabling "USB tethering" will get you a network over the USB cable.

If you're using Windows,

NFS from the phone: https://github.com/nekoni/nekodrive
SMB from the phone: http://windows.microsoft.com/en-us/windows-8/create-shortcut-to-map-network-drive

c627627 said:

The drive mounting yes can be done on certain phones, I have a program for Samsung Galaxy 4 but not for later modern models. I will look into your link and see if it works on my Samsung Galaxy 5. EDIT: wait, what did you use exactly to mount which phone to show up as a drive letter on PC?

Click to expand...

Step 1) Get WebDAV from the Playstore and let it install https://play.google.com/store/apps/details?id=com.theolivetree.webdavserver&hl=en

Step2) Press the button and get a local network IP (mine appeared to be the next available through DHCP 192.168.x.x:8080) If it isnt working maybe add a rule in your firewall.... Im not certain that is necessary though

Step 3) "Map a Network Drive" in windows and point it at the local address that webDAV gives you --> webDAV can be set to open any folder you choose, but automatically points to the default SD card location ( seems to vary by device, but was autodetected correctly on the 2 devices I tried it on) --> done. Open your "network drive/sdcard"

Note on the devices used: 1st try detected the mount point, but the device I used had no external sdcard , but DOES emulate one (/emulated_card/sdcard0 IIRC) could not connect to the network in any configuration I tried . This one was a Motorola MotoG

2nd device does have an external sdcard (I forget the mountpoint, but it was autodetected just fine) and worked nearly right away. I was afraid the connection had hung because it took longer than I thought it should but then *poof* there it was in Win7 mapped as a network drive. This was on a HTC Desire 510. I am unsure if HTCSync Manager has anything to do with success here, but does allow HTC phones to be recognized as a CD drive of some sort apparently. I usually remove it, but this phone was restored to factory settings a while ago, and I only pulled it out of the drawer for testing today.

Of course Wi-Fi is the problem here, WiFi is the limiting factor.

1. It has to be a cable connection through a mounted letter.
separate from that, separate issue:
2. Direct folder to folder sync on Android itself, external to internal microSD card sync.

Click to expand...

it is worth saying that there is an option in the settings of webDAV to choose a connection type (Wifi, Ethernet etc) and if you were to have usb/ethernet adapter it might work, but that is all guesswork on my part.

Everything you guys posted I will test and report back.

Thank you very much.

I am making progress with this. Thank you.
Mine was among the first households to get high speed internet 20 years ago but I've never needed more than 5 Mbps down and 1 Mbps up, which is what I still have....

I got it to work over WiFi but am experimenting how to do it with cable, since 1 Mbps is way too slow to sync a 10 GB folder.

So WiFi connection works but with 1 MBps upload speeds, it is less annoying to insert a microSD card and copy/paste the entire 10 GB folder then it is to wait for sync comparison to finish over a 1 MBps WiFi.

So when I use a USB cable to connect the device from USB port on Android cell / Android tablet to USB port on PC, can WebDAV server do over cable what it does over WiFi?


Here are WebDAV server Network Interface options, Wi-Fi works. Selecting any other does not work for USB to USB connection:




Would any of your links work for USB to USB, petteyg359?

c627627 said:

So WiFi connection works but with 1 MBps upload speeds, it is less annoying to insert a microSD card and copy/paste the entire 10 GB folder then it is to wait for sync comparison to finish over a 1 MBps WiFi.

So when I use a USB cable to connect the device from USB port on Android cell / Android tablet to USB port on PC, can WebDAV server do over cable what it does over WiFi?


Here are WebDAV server Network Interface options, Wi-Fi works. Selecting any other does not work for USB to USB connection:

View attachment 171996


?

Click to expand...

My guess is that you would need:

1) OTG compatability on the android device (not guaranteed, but not terribly rare either)

2) a usb -> ethernet adapter

Plug in the adapter to your ethernet port, attach the OTG and ZOOM!... maybe. Something tells me it might not work all pretty and nice like that. Sorry man, I thought this one was going into the solved column. Ill keep posting with new stuff as I find it. Im currently working with samba and backups and such, so this is right near those things in the technological department. Im going to try rsync next, as I beleive it should work seemlessly with a linux machine. I gather that you may prefer a Windows solution, but is it a "must have"?

I now believe this is not possible because cell phones and most tablets do not support USB to Ethernet, they simply don't.
My goal is not complicated, we are back to searching for a direct folder to folder sync app on Android, which once again, incredibly, does not exist.


If you hold a microSD card in the palm of your hand you can sync the contents quickly.... except on Android where it would be a thousand times slower in comparison.


Time for some out of the box thinking:

1. Create a new 10 GB folder on PC and copy it in its entirety to Android.

2. Instead of syncing that new 10 GB folder on Android, only compare it to source and copy ONLY the changed files to a third location on PC and then simply move that third location containing only the changed files to a microSD card for an easy and quick copy/paste on a mobile device.

c627627 said:

I now believe this is not possible because cell phones and most tablets do not support USB to Ethernet, they simply don't.

Click to expand...

To the contrary, practically all of them (at least Android, since at least 2.2) do. Just enable USB tethering, like I mentioned earlier, and run any kind of server on the phone. Sure, you're limited to talking with whatever computer the USB cable is plugged in to, but there's nothing stopping you building some kind of bridge device with an Arduino or Raspberry Pi and having it route between the network provided by a phone and your ethernet network.

Thethering is not available on reatail Samsung Galaxy and similar phones. You need to install a custom OS to have tethering and then that would be pointless since custom OS allows mounting SD cards as a drive letter thereby allowing super fast syncing anyway...

c627627 said:

Thethering is not available on reatail Samsung Galaxy and similar phones. You need to install a custom OS to have tethering and then that would be pointless since custom OS allows mounting SD cards as a drive letter thereby allowing super fast syncing anyway...

Click to expand...

There are apps that tether without root. ClockworkMod Tether worked VERY well for me with a few offbeat devices (LG Venice; Kyocera Hydro) and is worth atleast a cursory glance. I dont exactly recall but I think you might have to upgrade to a paid version eith that one.

Ive gotten other tether to work, but it was always more work than it was worth. The paid apps do the job right and right away (until android updates) The best tether app I ever had was ridiculously expensive and only worked for a few weeks before an update knocked it out. That's beside the point though. Tether can be had without root, and usually even if the carrier forbids it (sometimes they go to some pretty great extent to block it though- and succeed) The basic rule of thumb is that tethering WILL eat your plan data, which is what carriers are usually charging you for with tethering anyway, they give you an add on to your data coverage that would normally be outside your plan (why they do this I cannot even guess. Greed?). If you can live with the low speed data once you reach your monthly quota then that *seems* to be the only drawback.

I bought a brand new S5 not 2 months ago and it tethers fine. ***** of it is with lollipop there is no root.

Any results? I have to say I'm equally dumbfounded that I can't find an app that provides an interface for syncing 2 folders within an Android device.

My current situation is backing up my internal storage to my SD card. The copy stopped about 1000 files into 4000 and I have no way of finding out what's left to copy without exploring 157 subdirectories. That's an absurd task to do manually, and it's been solved programmatically in dozens of ways on other platforms. My best examples are WinMerge (folder diff including sub folders), and Filezilla FTP client with "synchronized browsing" and "file matching" enabled.

If you haven't found a solution yet, I'll personally make an Android app for this. It's simple path recursion, file listing, comparing source & destination, and providing some options of (mirror right from left / mirror left from right / copy extra from left / copy extra from right / copy all from each + ask user to handle conflicts on differences (overwrite with newest/oldest changed file, ask every time/ etc.) )

There are no words to describe how incredible it is that you appear to be the only person I ever came across who also thinks that it's more than strange that apparently no one out there has a need to sync two Android folders or else there would be an app for it and we are in April of 2017 and there is none (!!)

Please let me know if you start your project elsewhere. If you start it here, may I offer Beta testing feedback?

And now it's 2018. Any new developments?

I have a similar requirement (sync two folders locally), and am equally surprised that not only is there not a good solution for this, but people don't even seem to grasp the idea. It is indeed a common, and easily implementable task on every other computing platform.

Folder sync'ing is useful for a lot of things. *Some* of the specific use cases (such as manual backups) have other custom solutions for Android, but that's no reason to not still have the basic functionality.

What I'm looking for is basically unison for Android. Unison is a Linux too that utilizes rsync (I think) for two way, state aware, folder synchronizations. This includes propagating deletions from one sync'd folder to another etc. The two folders can be on the same drive, different drives on the same computer, or different computers.

However, I'd settle for a decent implementation of something like rsync on Android that I can then automate with Tasker. (And not requiring root).

EDIT:
I found this app "FolderSync Lite", which works for me. It's another cloud sync application, but it *does* let you choose a local folder for both the source and destination.

https://play.google.com/store/apps/details?id=dk.tacit.android.foldersync.lite