-
Notifications
You must be signed in to change notification settings - Fork 322
Enable multi-process commit mining in HTTP service #5388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Co-authored-by: suhaibmujahid <4151357+suhaibmujahid@users.noreply.github.com>
|
Let's wait a bit on this until the new deployment is fully functional. |
|
@suhaibmujahid we could now try to deploy this, but I would deploy it independently from other changes so we can easily revert it back should we run into issues. |
http_service/bugbug_http/models.py
Outdated
| revs=revs, | ||
| branch=repo_branch, | ||
| save=False, | ||
| use_single_process=True, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here we usually have very few patches to analyze, so might be worth keeping a single process? Or do we already do that by default when the number is small?
marco-c
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only one question to address
Since bugbug is no longer hosted on Heroku, memory constraints that required single-process commit mining are no longer relevant. This restores parallel processing for performance.
Changes
use_single_process=Truefromrepository.download_commits()calls in:http_service/bugbug_http/boot.py- commit DB updates during worker boothttp_service/bugbug_http/models.py- patch analysis for test selectionThe function defaults to
use_single_process=False, which enablesProcessPoolExecutorfor parallel commit and patch processing across all available CPU cores.Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.