Feature Requests

The MuWire file sharing application
User avatar
zlatinb
Posts: 105
Joined: 24 May 2018 17:14

Re: Feature Requests

Post by zlatinb »

zlatinb wrote: 25 Mar 2020 18:26
B0B wrote: 25 Mar 2020 18:17
zlatinb wrote: 25 Mar 2020 18:05 Can you go to /configlogging in the router console and enable logging at DEBUG level for the class "com.muwire.core.files.DirectoryWatcher".
That class doesn't exist. Closest (by name) is "com.muwire.core.files.PersisterFolderService".
Ok this is also a problem but I'm not going to open that can of worms until I figure out what is happening with your NAS. I'm going to write a small java program that mimics what MuWire does to watch for filesystem changes and post it here.
Here it is:

Code: Select all

import java.io.File;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;

import static java.nio.file.StandardWatchEventKinds.*;

public class WatchTest {

    public static void main(String[] args) throws Exception {
        if (args.length != 1) {
            System.err.println("Pass a directory to watch as an argument");
            System.exit(1);
        }
        
        File f = new File(args[0]);
        f = f.getCanonicalFile();
        
        Path p = f.toPath();
        
        WatchService watchService = FileSystems.getDefault().newWatchService();
        p.register(watchService, ENTRY_CREATE, ENTRY_MODIFY, ENTRY_DELETE);
        
        while(true) {
            WatchKey key = watchService.take();
            for (WatchEvent<?> event : key.pollEvents()) {
                if (event.kind() == ENTRY_CREATE)
                    System.out.printf("created %s %s\n",key.watchable(), event.context());
                else if (event.kind() == ENTRY_MODIFY)
                    System.out.printf("modified %s %s\n",key.watchable(), event.context());
                else if (event.kind() == ENTRY_DELETE) {
                    System.out.printf("deleted %s %s\n",key.watchable(), event.context()); 
                } else
                    System.out.println("???");
            }
            key.reset();
        }
    }

}
1. Save this in a file called "WatchTest.java"
2. Open a command prompt and type "javac -version". If that shows an error, then javac from AdoptOpenJDK isn't in your path, which is fine. It would be inside the bin\ folder in the jdk installation
3. Compile the code above by typing "path\to\javac.exe WatchTest.java"
4. Execute the program by typing "path\to\java.exe -cp . WatchTest \path\to\watch"

Then try dropping and deleting files in the \path\to\watch. The program should print something every time you drop or remove a file. If it doesn't, then we'll have to try different version of Java or start looking at the NAS drivers, or something along those lines.
GPG: https://keybase.io/zlatinb
blog: http://zab.i2p
mail: zab@mail.i2p
MuWire: https://muwire.com
MuCats: http://mucats.i2p
User avatar
B0B
Posts: 46
Joined: 18 Mar 2020 11:48
Location: NL

Re: Feature Requests

Post by B0B »

zlatinb wrote: 25 Mar 2020 18:45 1. Save this in a file called "WatchTest.java"
2. Open a command prompt and type "javac -version". If that shows an error, then javac from AdoptOpenJDK isn't in your path, which is fine. It would be inside the bin\ folder in the jdk installation

Java 8 is in my path but I2P is configured to use v13 so I need to run it from \bin\ which will do.
javac 13.0.2
zlatinb wrote: 25 Mar 2020 18:45 4. Execute the program by typing "path\to\java.exe -cp . WatchTest \path\to\watch"
bin\java.exe -cp . WatchTest.java \\NAS3\ide4\MuWireWatchTest
???
???
???


When adding files, nothing will be printed.
When deleting files, 3 questionmarks appear (added and deleted 3 files)
zlatinb wrote: 25 Mar 2020 18:45 The program should print something every time you drop or remove a file. If it doesn't, then we'll have to try different version of Java or start looking at the NAS drivers, or something along those lines.
Note that the NAS' I have are not using specialized drivers. Those run an embedded linux and are accessed (samba) like any other networked device.

edit:
Did the same test again with using the mapped device ID (drive letter) in stead of network path and the results are exactly the same.

edit 2:
Moved the test folder to D:\ and repeated above. Results prove your test code is o.k.

bin\java.exe -cp . WatchTest.java D:\MuWireWatchTest
created D:\MuWireWatchTest Jefferson Airplane - White Rabbit [lyrics].txt
modified D:\MuWireWatchTest Jefferson Airplane - White Rabbit [lyrics].txt
modified D:\MuWireWatchTest Jefferson Airplane - White Rabbit [lyrics].txt
created D:\MuWireWatchTest Judas Priest - Breaking the Law [lyrics].txt
modified D:\MuWireWatchTest Judas Priest - Breaking the Law [lyrics].txt
modified D:\MuWireWatchTest Judas Priest - Breaking the Law [lyrics].txt
created D:\MuWireWatchTest The Music - Getaway [lyrics].txt
modified D:\MuWireWatchTest The Music - Getaway [lyrics].txt
modified D:\MuWireWatchTest The Music - Getaway [lyrics].txt
deleted D:\MuWireWatchTest Jefferson Airplane - White Rabbit [lyrics].txt
deleted D:\MuWireWatchTest Judas Priest - Breaking the Law [lyrics].txt
deleted D:\MuWireWatchTest The Music - Getaway [lyrics].txt
User avatar
zlatinb
Posts: 105
Joined: 24 May 2018 17:14

Re: Feature Requests

Post by zlatinb »

Found this https://stackoverflow.com/questions/847 ... ped-drives

Basically a java bug and it doesn't look like it's going to get fixed. Only solution is to write custom polling. I can't make any promises as to when I can deliver that, although I understand it's necessary.
GPG: https://keybase.io/zlatinb
blog: http://zab.i2p
mail: zab@mail.i2p
MuWire: https://muwire.com
MuCats: http://mucats.i2p
User avatar
B0B
Posts: 46
Joined: 18 Mar 2020 11:48
Location: NL

Re: Feature Requests

Post by B0B »

zlatinb wrote: 25 Mar 2020 19:52 Basically a java bug and it doesn't look like it's going to get fixed. Only solution is to write custom polling. I can't make any promises as to when I can deliver that, although I understand it's necessary.
Well, at least the problem is pinpointed. That's a start.

What's also possible is a manual refresh of the shared files. Some P2P programs use their own built-in polling routines but also have an option to activate it manually. I think MuWire already has, more or less, something like that because files can be refreshed when un-sharing/re-sharing a folder. Although... I should test that first...

edit:
Tested. It works but needs to rehash everything again. When not erasing the existing hashes in between the un-share/re-share action it can be used as an alternative to polling. You can also trigger those routines in your java code and add an poll interval value to MuWire.properties. Could be I'm simplifying it too much.
User avatar
zlatinb
Posts: 105
Joined: 24 May 2018 17:14

Re: Feature Requests

Post by zlatinb »

B0B wrote: 25 Mar 2020 20:12 What's also possible is a manual refresh of the shared files. Some P2P programs use their own built-in polling routines but also have an option to activate it manually. I think MuWire already has, more or less, something like that because files can be refreshed when un-sharing/re-sharing a folder. Although... I should test that first...

edit:
Tested. It works but needs to rehash everything again. When not erasing the existing hashes in between the un-share/re-share action it can be used as an alternative to polling. You can also trigger those routines in your java code and add an poll interval value to MuWire.properties. Could be I'm simplifying it too much.
It's going to end up something along those lines, but there are many corner cases I need to cover so it's going to take a while to do it right. I'll add it to the TODO.md list.
GPG: https://keybase.io/zlatinb
blog: http://zab.i2p
mail: zab@mail.i2p
MuWire: https://muwire.com
MuCats: http://mucats.i2p
User avatar
B0B
Posts: 46
Joined: 18 Mar 2020 11:48
Location: NL

Re: Feature Requests

Post by B0B »

zlatinb wrote: 25 Mar 2020 20:26 It's going to end up something along those lines, but there are many corner cases I need to cover so it's going to take a while to do it right. I'll add it to the TODO.md list.
I'm sure you'll find a way to do it. Next week my time is much more limited. Was at home due to the current situation but work is calling again so I'll be less visible over here. Or anywhere.

For now I know how I can get around it, which is by just un-/re-sharing new folders.
User avatar
zlatinb
Posts: 105
Joined: 24 May 2018 17:14

Re: Feature Requests

Post by zlatinb »

I'm going to make every watched directory configurable separately from the others. There will also be a default value.

I will need an icon for an "Advanced Sharing" page in the "Configuration" section on the sidebar. On that page there will be a table with all watched directories and ability to configure refresh interval (and other settings that may come up in the future) for each watched directory.

Also, in the Shared Files Tree there will be a "Re-scan" option in the hover menu which does a one-time re-scan.
GPG: https://keybase.io/zlatinb
blog: http://zab.i2p
mail: zab@mail.i2p
MuWire: https://muwire.com
MuCats: http://mucats.i2p
danrobi9781
Posts: 1
Joined: 23 Mar 2020 21:37

Re: Feature Requests

Post by danrobi9781 »

@BOB Good job with the new MW icons :D (A bit late cuz registration process)
User avatar
B0B
Posts: 46
Joined: 18 Mar 2020 11:48
Location: NL

Re: Feature Requests

Post by B0B »

zlatinb wrote: 25 Mar 2020 21:18 I'm going to make every watched directory configurable separately from the others. There will also be a default value.
Sounds promising.
zlatinb wrote: 25 Mar 2020 21:18 I will need an icon for an "Advanced Sharing" page in the "Configuration" section on the sidebar.
Done. Want to give the git a try. I guess I need to send a "request" notice? (which I did).
danrobi9781 wrote: 25 Mar 2020 21:25 @BOB Good job with the new MW icons :D (A bit late cuz registration process)
Thanks. After all, I'm also benefiting from it :-)
User avatar
zlatinb
Posts: 105
Joined: 24 May 2018 17:14

Re: Feature Requests

Post by zlatinb »

B0B wrote: 25 Mar 2020 22:38
zlatinb wrote: 25 Mar 2020 21:18 I will need an icon for an "Advanced Sharing" page in the "Configuration" section on the sidebar.
Done. Want to give the git a try. I guess I need to send a "request" notice?
What you want to do is "Fork" the repo, then "Clone" from your fork (which will appear under your namespace). Then commit the new icon to your fork, (in "webui/src/main/images/" then create a merge request back to my repo.

You should have your account enabled by now, if it isn't let me know I'll pester idk. He wrote the following guide https://geti2p.net/en/blog/post/2020/03/06/git-over-i2p (look at "Suggested workflow for developers"). If you're on windows, I suggest trying out https://gitforwindows.org
GPG: https://keybase.io/zlatinb
blog: http://zab.i2p
mail: zab@mail.i2p
MuWire: https://muwire.com
MuCats: http://mucats.i2p
Post Reply