Download Word

[INFRA-32] Scrub drp_stella for old large files Created: 18/Jun/16
Updated: 16/Jul/16 Resolved:
16/Jul/16
Status:
Project:
Component/s:
Affects
Version/s:
Fix Version/s:
Done
Software Development Infrastructure
None
None
Type:
Reporter:
Resolution:
Labels:
Remaining
Estimate:
Time Spent:
Original
Estimate:
Task
rhl
Done
None
Not Specified
Issue Links:
Relates
relates to
Sprint:
Reviewers:
None
Priority:
Assignee:
Votes:
Major
aritter
0
Not Specified
Not Specified
PIPE2D-37
Justify detector geometry constants
Done
2014-13, 2014-14
swinbank
Description
(e.g. ipython notebooks). N.b. google for git-filter-branch or use "BFG repo cleaner"
Comments
Comment by aritter [ 26/Jun/16 ]
Hmmm... While bfg worked for the ".fits" files in the "tests" directory, it doesn't seem to work
for ".fits.gz" files nor for the ".ipynb" files. Although bfg says that it deleted the files they are
still there. Gonna have to try something else...
Comment by aritter [ 28/Jun/16 ]
So apparently this worked just fine:
git filter-branch --force --index-filter 'git rm --cached -r --ignore-unmatch
python/pfs/drp/stella/*.ipynb' --prune-empty --tag-name-filter cat – --all
git filter-branch --force --index-filter 'git rm --cached -r --ignore-unmatch
tests/data/calExp/2016-01-12/v000000*/PFFAr2.fits.gz' --prune-empty --tag-name-filter cat – --
all
git reflog expire --expire=now --all
git gc --prune=now
git gc --aggressive --prune=now
git push origin master --force
Comment by aritter [ 28/Jun/16 ]
All fits files and not needed ipython notebooks have been removed from the repo
Comment by rhl [ 28/Jun/16 ]
All ... and not needed ipython notebooks have been removed from the repo
Which notebooks are needed?
Comment by aritter [ 28/Jun/16 ]
createDetGeom.ipynb to create the detector geometry fits files
and
createRefSpec.ipynb to create the reference spectrum
Comment by rhl [ 28/Jun/16 ]
I don't think any executable scripts should be notebooks, please export them as straight python,
and make sure that they can be run from the command line (by adding argparse argument
parsing etc.).
If you have educational notebooks, please move them into a new product.
Comment by aritter [ 28/Jun/16 ]
"createDetGeom" has been moved to a task to be executed from bin.src/createDetGeom.py
(command line task didn't work because there are no data to be processed and no butler to be
associated with. Error: "WARNING: Not running the task because there is no data to process;
you may preview data using "--show data"")
"createRefSpec" has been moved to a command line task
See drp_stella tickets/
Comment by swinbank [ 02/Jul/16 ]
Comments on the code at https://github.com/Subaru-PFS/drp_stella/pull/1 – I think a bit more
thought could make the scripts a clearer, but I don't think there are any obvious regressions here,
so we should probably file another issue for refactoring.
Confirmed that drp_stella is now pruned to a sensible size.
Comment by swinbank [ 16/Jul/16 ]
Will address provenance of detector geometry in . Other comments have been addressed.
Generated at Mon May 15 00:20:47 JST 2017 using JIRA 7.1.8#71012sha1:d83491778ddf5df967f3c738fa396f8b341ab597.