Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
[INFRA-32] Scrub drp_stella for old large files Created: 18/Jun/16 Updated: 16/Jul/16 Resolved: 16/Jul/16 Status: Project: Component/s: Affects Version/s: Fix Version/s: Done Software Development Infrastructure None None Type: Reporter: Resolution: Labels: Remaining Estimate: Time Spent: Original Estimate: Task rhl Done None Not Specified Issue Links: Relates relates to Sprint: Reviewers: None Priority: Assignee: Votes: Major aritter 0 Not Specified Not Specified PIPE2D-37 Justify detector geometry constants Done 2014-13, 2014-14 swinbank Description (e.g. ipython notebooks). N.b. google for git-filter-branch or use "BFG repo cleaner" Comments Comment by aritter [ 26/Jun/16 ] Hmmm... While bfg worked for the ".fits" files in the "tests" directory, it doesn't seem to work for ".fits.gz" files nor for the ".ipynb" files. Although bfg says that it deleted the files they are still there. Gonna have to try something else... Comment by aritter [ 28/Jun/16 ] So apparently this worked just fine: git filter-branch --force --index-filter 'git rm --cached -r --ignore-unmatch python/pfs/drp/stella/*.ipynb' --prune-empty --tag-name-filter cat – --all git filter-branch --force --index-filter 'git rm --cached -r --ignore-unmatch tests/data/calExp/2016-01-12/v000000*/PFFAr2.fits.gz' --prune-empty --tag-name-filter cat – -- all git reflog expire --expire=now --all git gc --prune=now git gc --aggressive --prune=now git push origin master --force Comment by aritter [ 28/Jun/16 ] All fits files and not needed ipython notebooks have been removed from the repo Comment by rhl [ 28/Jun/16 ] All ... and not needed ipython notebooks have been removed from the repo Which notebooks are needed? Comment by aritter [ 28/Jun/16 ] createDetGeom.ipynb to create the detector geometry fits files and createRefSpec.ipynb to create the reference spectrum Comment by rhl [ 28/Jun/16 ] I don't think any executable scripts should be notebooks, please export them as straight python, and make sure that they can be run from the command line (by adding argparse argument parsing etc.). If you have educational notebooks, please move them into a new product. Comment by aritter [ 28/Jun/16 ] "createDetGeom" has been moved to a task to be executed from bin.src/createDetGeom.py (command line task didn't work because there are no data to be processed and no butler to be associated with. Error: "WARNING: Not running the task because there is no data to process; you may preview data using "--show data"") "createRefSpec" has been moved to a command line task See drp_stella tickets/ Comment by swinbank [ 02/Jul/16 ] Comments on the code at https://github.com/Subaru-PFS/drp_stella/pull/1 – I think a bit more thought could make the scripts a clearer, but I don't think there are any obvious regressions here, so we should probably file another issue for refactoring. Confirmed that drp_stella is now pruned to a sensible size. Comment by swinbank [ 16/Jul/16 ] Will address provenance of detector geometry in . Other comments have been addressed. Generated at Mon May 15 00:20:47 JST 2017 using JIRA 7.1.8#71012sha1:d83491778ddf5df967f3c738fa396f8b341ab597.