CanuteTheGreat/sgspider
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
No description
16 commits 1 branch 0 tags 64 KiB
Find a file
Exact
Ronald Farrer 9bb74f8015 Add file validation, memory management, GPU disabling, and instance locking 
- Add file validation to detect corrupted/placeholder images via magic bytes
- Fix download errors by using context.request.get() with response.dispose()
- Add periodic browser restart to manage memory (configurable interval)
- Add comprehensive GPU-disabling flags to prevent NVIDIA memory exhaustion
- Add file locking to prevent multiple instances from running simultaneously
- Add command-line argument support for processing specific album URLs
- Lower minimum file size threshold from 100KB to 10KB
- Update .gitignore for lock file and __pycache__

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-12 16:44:48 -08:00
.gitignore Add file validation, memory management, GPU disabling, and instance locking 2025-12-12 16:44:48 -08:00
sgspider.example.ini Add file validation, memory management, GPU disabling, and instance locking 2025-12-12 16:44:48 -08:00
sgspider.py Add file validation, memory management, GPU disabling, and instance locking 2025-12-12 16:44:48 -08:00