Enhancement: File Handling and Compression Support #4

sad · 2024-02-28T13:39:43Z

sad commented

2024-02-28 13:39:43 +00:00

Summary:
This pull request introduces significant enhancements to the file upload functionality, including support for gzip compressed files and improvements to the file handling process in the UploadHandler function. Additionally, it extends the client-side functionality to allow for file compression before upload, leveraging the pako compression library.

Changes:
Server-Side

Imported compress/gzip, strings, and bytes packages to support gzip
Added isValidGzipHeader function to validate gzip headers in uploaded
Modified UploadHandler
Buffer file uploads for Decompress gzip files with a .5000 extension if they have a valid gzip
Directly handle non-gzip files and provide detailed error
Use MIME type detection on processed data to accommodate both compressed and uncompressed files.
Improved file writing with os.O_TRUNC to replace contents on upload.

Client-Side Enhancements:

Integrated the pako compression library for client-side gzip compression of files before uploading.
Updated file selection script to compress files, add a .5000 extension for compressed files, and handle UI feedback.

Testing:

Thorough testing is recommended for both compressed and uncompressed file uploads to validate the new functionality.
Verify client-side compression results in correctly formatted gzip files. Test Multiple File Types
Ensure server-side decompression handles various gzip files correctly and falls back gracefully for non-gzip or corrupt inputs.
Confirm that MIME type detection accurately identifies file types post-compression or decompression.

Solves: #3 (comment)

**Summary:** This pull request introduces significant enhancements to the file upload functionality, including support for gzip compressed files and improvements to the file handling process in the UploadHandler function. Additionally, it extends the client-side functionality to allow for file compression before upload, leveraging the pako compression library. **Changes:** _**Server-Side**_ - Imported `compress/gzip`, `strings`, and `bytes` packages to support gzip - Added `isValidGzipHeader` function to validate gzip headers in uploaded - Modified `UploadHandler` - Buffer file uploads for Decompress gzip files with a `.5000` extension if they have a valid gzip - Directly handle non-gzip files and provide detailed error - Use MIME type detection on processed data to accommodate both compressed and uncompressed files. - Improved file writing with `os.O_TRUNC` to replace contents on upload. _**Client-Side Enhancements:**_ - Integrated the `pako` compression library for client-side gzip compression of files before uploading. - Updated file selection script to compress files, add a `.5000` extension for compressed files, and handle UI feedback. **Testing:** - Thorough testing is recommended for both compressed and uncompressed file uploads to validate the new functionality. - Verify client-side compression results in correctly formatted gzip files. _Test Multiple File Types_ - Ensure server-side decompression handles various gzip files correctly and falls back gracefully for non-gzip or corrupt inputs. - Confirm that MIME type detection accurately identifies file types post-compression or decompression. **Solves**: https://git.supernets.org/supernets/hardfiles/issues/3#issue-11_

sad added the

enhancement

label 2024-02-28 13:39:43 +00:00

sad added 1 commit 2024-02-28 13:39:44 +00:00

implemented compression 8e65053cc1

sad requested review from delorean 2024-02-28 13:39:53 +00:00

sad requested review from acidvegas 2024-02-28 13:39:53 +00:00

sad commented

2024-02-28 13:41:47 +00:00

Most affected will be front-end users to try the changes via curl make sure to gzip your file and replace gz with 5000

Most affected will be front-end users to try the changes via curl make sure to gzip your file and replace `gz` with `5000`

sad requested review from hgw 2024-02-28 13:42:31 +00:00