General > Developers Corner

Sending large files from server to client

(1/2) > >>

Timmy:
Context
I’m working on a module that displays ULX logs in XGUI.

Problem
• Active servers can have large log files (up to several megabytes).
• All data transfers in Garry’s Mod are throttled to 20 kb/s.
• A Net message can only contain up to 64 kb of data.

Current approach
Send compressed contents with xgui.SendDataEvent
+ Simple
- Fails when dealing with large log files

Alternative approaches
"Load more…" button
Send a snippet. Make user press button to load the next ~64 kb.
+ Fast, works with large files
- Clutters UI, inconvenient when dealing with large files

Send in chunks
Split contents into small chunks. Send all the chunks.
+ Works with large files
- Very slow and intense lag when dealing with large files

Remote server
Upload log to a remote server.
+ Access to all of the contents, fast
- Server can go offline, client may not have internet access, access to host may be blocked
Are there any other approaches I should consider? How would you send large files to users?

JamminR:
Though still a type of "remote server", don't host remotely.
I just came up with this idea,and apparently other people have already made it happen.
Lua web server. (Does Gmod Awesomium limit file size?
Several exist.
https://www.google.com/search?q=lua+web+server
I wasn't overly surprised, as I use a Python CherryPy/Mako based web server, but, yeah, Lua.

Timmy:
Thanks, JamminR! I appreciate your input.

Though none of those web servers are written in pure Lua. They all require binary modules written in C/C++. I don’t believe it is possible without those. :(

There is a networking module for Garry’s Mod written by a third party called Bromsock.

I expect this would be fast and work with large files.

But this would reduce accessibility as server owners would have to manually install (and update) the binary module. They would also have to update their firewall to allow traffic on an additional port.

I would also be relying on the maintainer’s ability to keep the project up-to-date and secure.

JamminR:
Welcome. I didn't look that deep - figures - I'd hoped something would be available in pure lua.
Loading binary in this instance is cheating! :P :D

MrPresident:
You could use an approach I used for sending screenshots from the client to the server. Should work backwards.

Take the log file and convert it to binary, then break it up into chunks and send a net message to the user with a header of the current chunk and the total chunks. Then have the client wait until it receives all the chunks and put them back together.

Navigation

[0] Message Index

[#] Next page