Download Program Review and Action Planning – YEAR THREE Final Summary Report

Program Review and Action Planning – YEAR THREE
Final Summary Report
Division
Program
Contact Person
Date
I.
Science & Math
Astronomy
Scott Hildreth & Tim Dave
March 2011
Reflect upon the last three years' analysis and activities.
Our 2009 Program review concluded that, in the next review cycle, we would focus upon two key areas:
1) Assessing and Evaluating student success in our astronomy classes; and,
2) Exploring how best to take advantage of new facilities and new equipment provided by the
Measure B Bond.
Key Program Review Focus Area #1: Assessing and Evaluating Student Success in Astronomy
Our 2009 program review looked both “inward”, comparing student success across the campus with that
achieved in our astronomy lecture classes, and “outward” to community colleges across the Bay Area region,
seeking to understand whether Chabot’s students were succeeding at a different rate than other schools. From
the external data available at the time, Chabot’s students were succeeding in our two astronomy lecture classes
at the same rate, or below that of other nearby schools – but this was a very limited survey. Since that time, the
Chancellor’s office makes program statistics available online for success rates by college, and with a few hours
of research, Chabot’s astronomy program success can be ranked relative to other colleges:
College
Chabot
Las Positas
Ohlone
San Mateo
Diablo Valley
Foothill
Contra Costa
Hartnell
De Anza
Avg Greater SF
Bay Area
Statewide
Fall 2010
Success
53.5%
53.6%
38.3%
no data available
no data available
no data available
no data available
no data available
no data available
no data
available
Summer 2010
Spring 2010
Fall 2009
Spring 2009
62%
52.6%
60.3%
80.2%
81.3%
no data available
no data available
no data available
no data available
56.5%
54.6%
40.4%
70.7%
68.1%
84.1%
54.2%
74.2%
79%
55.6%
48.7%
47.7%
68.7%
64.9%
69.3%
50%
85%
82%
53.5%
43.6%
41.5%
73.4%
68.2%
81.9%
63.3%
77.6%
82%
77%
64.7%
64.5%
62%
63%
63%
Source: California Community Colleges Chancellors Office. (2011) Program Retention/Success Rates For Credit Enrollments By Distance
Education Status. Sub Discipline Astronomy (Program Code 1911). Accessed from
https://misweb.cccco.edu/mis/onlinestat/ret_sucs_de_rpt.cfm?timeout=800.
The success rate for Chabot students in Astronomy is comparable to that of students taking Bio 31, another
introductory science course with no prerequisites, showing an average success rate for the past six full semesters
from available institutional data as 53%. Presumably the same types of students take both courses.
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Page 1
Assessing and Evaluating Student Success in Astronomy (cont.)
It is apparent that, looking at Astronomy program success scores only, Chabot indeed ranks below both Bay
Area averages, and statewide averages. These results help to frame a larger inquiry that could form our next
program review cycle – contacting the schools that are doing even better than Chabot, and looking critically at
what they are doing to achieve the success rates they report. For example, Santa Barbara College shows both
one of the largest astronomy programs (three to four times larger in enrollment each term than Chabot), and one
of the highest success rates, in the state:
College
Santa Barbara
Fall 2010
Success
79.3%
Summer 2010
Spring 2010
Fall 2009
Spring 2009
89.3%
79.0%
78.1%
79.9%
Are these success results due solely to different demographics? Or are they a result of larger lecture delivery
and assessment modes compared with essay exams administered at Chabot by some faculty. Locally colleges
like De Anza, Foothill, and Marin have higher success rates than Chabot for their Astronomy programs, but
again demographics might logically be one reason for that difference. Still, we can reach out and see what
factors those schools might be able to identify as key reasons for their success, and learn from that work.
Still, success by grades (A,B, C, or Pass) is just one measure of achievement. It will be important to use
additional assessments through the SLO process to gauge student success on other terms. Using the CLO
process, we are looking at doing even more assessment of student success in our classes across a variety of
metrics, including appreciation for the process of science as illustrated in recent Astronomical discoveries, as
well as appreciation for Astronomy’s impact on our students’ lives. We might be able to look at CLO’s of other
schools, and compare how students are doing with identical assessments.
Key Program Review Focus Area #2: Exploring how best to take advantage of new facilities and equipment
This area was indeed the major focus of our time and energy over the past 3 years. The renovation of Building
1900, the move of all planetarium equipment to the Physics lab, teaching Astronomy in the first generation of
“smart” classrooms, and the subsequent move back to the new planetarium, consumed enormous amounts of
faculty time, far beyond the hours associated contractually for professional collegial participation. The
planetarium did indeed come online with new equipment in Summer 2010, and we have begun to learn how to
use the new projector, as well as all of the new AV equipment. All lecture classes are being held in the
planetarium for both Autumn 2010 and Spring 2011 terms, and students are very positive in their enjoyment and
appreciation of the facilities. We would like to investigate retention and student success data from Chabot’s
database over the next three years, to learn whether the new facilities might indeed be identified as playing a
role in any improvements.
As with any advanced technology, we have encountered numerous problems with the sound system, AV system
programming, and Software. We are spending at least 5-10 hours a week working with the systems, and much
more time developing public programs, learning to program the system for more automated operations. We’ve
continued our training with the system, including 6 additional hours in Spring 2011 from Spinitar.
With the challenges presented by the new lecture halls and planetarium equipment, and the required
involvement of our two full-time faculty in the new Building 1800 project, we’ve not had the time to install and
begin to use the 12” optical telescope and portable observatory dome.
Our goals continue to be learning how to integrate this equipment into both our lecture and laboratory classes, as
well as for use with public shows. Public events will start that term with a grand opening, followed by a
schedule of community and school shows over the academic year.
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Page 2
II. Briefly summarize the accomplishments of the discipline, and how they relate to the review of
the program, the program-level outcomes (PLOs) and course-level outcomes (CLOs).
Our largest accomplishment in the program review cycle was re-establishing our planetarium as a
state-of-the art lecture theater. We also made progress implementing SLO/CLO’s for our courses,
integrating new labs, and testing out new curriculum delivery options with a year-long field test of
a new textbook and online learning system offered by WH Freeman with four astronomy classes.
The table below summarizes our continuing Astronomy Program Goals from the Year Two
Program Review (March 2010), and notes the overall achievements for each major goal.
Astronomy Program Goals from March 2010 Program
Review Unit Plan
Overall Achievement
1. Continue to assess student learning and success in both
Astronomy lecture and lab courses, using SLOs and online
systems.
Ongoing
2. Attend planetarium training for the new projector systems in
April and June 2010; receive AV systems training in March &
April 2010. Explore how the new planetarium equipment
positively affects student success and satisfaction, as well as
campus visibility in the community
Planetarium now in operation for
all classes; assessments for
student success starting
3. Explore how new laboratory telescopes and equipment can be
integrated into the laboratory courses.
4. Develop the pool of qualified applicants for adjunct teaching
opportunities
5. Monitor the support time and costs required for maintaining the
new planetarium projector and AV equipment, so that the
college can adequately budget for the future. We face a
preventative maintenance bill for the new projector of about
$6-7K a year – almost double the old amount; in addition, we
face another maintenance agreement cost of about $4K per
year for the new digital AV systems.
6. Explore how to offer public programs at the planetarium, for
the community and local schools, and fund the acquisition
and/or development of new programs to attract the community
and the student to Chabot.
7. Continue to lead the Building 1700-1800 user groups, specifically
with the design of the physics labs, tutorial spaces, and
storage.
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Still in progress
Stopped due to limited FTEF
allocation and continued program
cuts – no classes to offer.
Warranty for the equipment lasts
until the end of February 2012; by
then we need to know if we can
afford, or afford not, to contract
for support.
Public Programs have started on a
very limited basis. They can be
extended with the hiring of
additional program help, planned
for Autumn 2011.
Ongoing.
Page 3
The table below details the continuing program activities in support of these goals from March 2010,
updated with our current achievements. We were successful with every major activity save for
constructing a new portable planetarium and telescope, integrating its use into the lab curriculum, and
developing more public and school programs for the new planetarium. It is important to note that the
development and delivery of public programs are outside of our teaching and collegial professional
loads, compensation, and hours. An additional spreadsheet with greater detail is also attached at the
end of this report.
Spring 2010:







Year 2 -3 Program Review Activites
Complete 1902 Renovation
Training on AV systems: March/April 2010
3-Day intensive training on planetarium systems: April 5-7
Install and customize planetarium instructional computer system
Use Planetarium for special class visits for current Spring 2010 classes: May
(if/when occupancy is granted)
Continue SLO development and assessment cycle
Continue to assist in the definition of building 1700 and 1800
Status
All
Successfully
completed
Summer 2010:





Teach initial class in the planetarium
Develop plans for grand opening
Develop plans for portable observatory
Continue SLOs
Continue to assist in the definition of building 1700 and 1800
All
Successfully
completed
Autumn 2010:




Teach all lecture classes in planetarium
Develop initial public event
Continue SLOs
Continue to assist in the definition of building 1700 and 1800


Revise Lab curriculum to utilize new telescopes and equipment
Develop initial school shows
Successfully
completed
Not yet
completed
Spring 2011:





Continue teaching all lecture classes in planetarium
Continue initial public shows
Assess first-year maintenance/operations costs for inclusion into subsequent year
budget planning.
Continue SLOs
Continue to assist in the definition of building 1700 and 1800


Continue ischool shows
Test lab curriculum with new telescopes
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Successfully
completed
Not yet
completed
Page 4
III. Please list what best practices (e.g., strategies, activities, intervention, elements, etc.) you would
recommend? What was challenging? Was there a barrier(s) to success?
Best practices to recommend:
1. Faculty Involvement in Building Renovation
From our work on the Building 1900 renovation (stretching more than 4 years), and the current
work on Building 1700/1800 renovation, we have learned that faculty participation cannot stop
with the initial user definitions. Given the specialized nature of the planetarium, we were
called on almost daily to help decide construction dilemmas, call attention to details
overlooked, review proposed changes, and explore new options. We expect similar
requirements will be present in the construction of the new physics labs in 1800. The college
should recognize that the best way to get the very best buildings built, serving the needs of the
students and the faculty working within them, greater faculty participation is required – and
should be reflected with at least some reassignment time. To expect faculty to do their ongoing
teaching, committee, and professional work as well as handle building issues will not produce
as much success.
2. Increased Local Faculty Responsibility for Maintenance of Office/Lab Computer Resources
Over the last program review cycle, we’ve seen numerous problems with our
Physics/Astronomy wireless system in Building 1700, as well as challenges getting software
installed on lab computers that have long since been used beyond their intended lifetime. In
the same time, online learning systems provided by our textbook publishers have exploded in
sophistication and power – but they, too, often require installation of new java, flash, and other
plug-ins. As more and more computers are deployed on campus – without an increase in
Chabot’s local computer support team – we face longer delays in getting equipment installed,
and longer wait times for necessary software upgrades, unless we can learn to be more
effective. One way to do that would be to grant selected users local administrative authority,
which could be used to install simple plug-in’s and publisher software. With suitable training,
and under the proper guidance of Chabot’s local team, a faculty team could speed response and
provide greater local help to colleagues.
Challenges/Barriers to Success:
1. Insufficient time for faculty to provide substantial comments to the building
project management team. On many occasions in the development of our
Building 1900 project, we were provided with very limited windows to
comment upon new plans, with an implied threat that if we made no comments
by a deadline, the plans would go ahead as is. We need a more reasonable
opportunity review and discuss changes that affect our teaching and ultimately,
our students.
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Page 5
Challenges/Barriers to Success (cont.)
2. Implementing SLOs, CLOs, and PLOs has been made extremely complex with
the required use of eLumen. Only with the tremendous support of our campus
webmaster, Abdullah Yahya, were we finally able in 2010 to use a simple webbased interface for the creation of new CLOs and PLOs – but unfortunately
doing that work did not automatically translate into those objectives being
included into the eLumen system for reporting. If we really want faculty to use
the CLO/PLO process authentically, the interface and reporting needs major
revision, and a simple web-based system for data entry and reporting must be
devised.
3. The lack of a laboratory assistant, or lab technician who can assist faculty in the
maintenance of the physics/astronomy lab and in setting up experiments
continues to be a huge barrier to our program success. When physics and
astronomy faculty must spend hours setting up and taking down labs –
something that other science faculty do not have to do at Chabot, nor at LPC, we
are consequently not able to spend that time improving our teaching and helping
our students learn.
4. Establishing planetarium school programs, and a more vibrant community
program, requires more help. Faculty already donate their time and energy for
community education and special programs held on Friday and Saturday nights.
Developing a new community program with our equipment requires additional
staff help, along the lines of what Hartnell College does with its paid
planetarium educator, Andy Kreyche.
IV. Next Steps: Recommendations for program and institutional improvement.
Program Improvement:
1. Hire an adjunct faculty colleague trained in the use and development of programs for digital
planetarium projection. Develop community outreach planetarium programs that could be
offered at least monthly, as well as limited school shows. For a very modest attendee cost at
these programs, the time of that faculty member could be largely offset, and their work
leveraged using it within our daytime classes.
2. Hire a student lab assistant, or part-time lab coordinator to help with the inventory and
preparation for moving the physics labs to the new Building 1800. As soon as such a person
could be found, they could help faculty set up and take down labs, identify equipment no longer
being used (and available for surplus, saving the cost to move it to the new lab space.)
3. Fund attendance for at least one faculty member to the Spitz Institute training offered annually
in Pennsylvania each summer, so that we might learn how to use our system even more
effectively.
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Page 6
Goal
#
Original Timeline
from March 2010
Unit Plan Program
Review (Year 2)
1
Spring 2010
2
Chabot College Astronomy Program
Person(s)
Responsible
Milestone Activity
Accomplished?
Yes/No/
In Progress*
Reviewed
March 2011
Revised?
Yes / No
If yes, list
revision year
Do you need
additional funds
to support this
activity?
Yes/No
If, yes, what
type?**
Hildreth/Dave
Yes
Spring 2010
Complete 1902 Renovation
Begin using 1904, 6, 8 Lecture Halls for Astronomy & Physics lectures
Training on AV systems/ Customizing Audio systems
Hildreth/Dave
no
3
Spring 2010
Begin training on use of new Spitz SciDome planetarium
Hildreth/Dave
Yes
100 + person
hours spent to
date on
AudioVisual
training
Yes
4
Spring 2010
Continue to work on Building 1700-1800 Definition & design
Hildreth/Dave
Yes
YES travel
5
Spring 2010
Use Planetarium for current class field trips if possible
Hildreth/Dave
Yes
no
6
Summer 2010
Begin installation of 12” Reflector and Portable Dome
Hildreth Dave
Still in progress
no
7
Summer 2010
Begin using Planetarium for one class as a pilot
Hildreth
Yes
no
8
Summer 2010
Develop Plans for Grand Opening
Hildreth/Dave
9
Autumn 2010
Begin using Planetarium for all astronomy lecture classes
Dave/Hildreth
Yes
10
Autumn 2010
Begin developing how to use new telescopes in Astronomy Laboratory
Hildreth/Dave
No
11
Autumn 2010
Start initial public shows in planetarium for community & schools
Hildreth/Dave
Yes
YES
12
Autumn 2010
Begin exploring how to offer community nights with new planetarium
Hildreth/Dave
Still in progress
13
Spring 2011
Begin student labs with new telescope equipment in Astro 30
Hildreth/Dave
Still in progress
YES staffing,
supplies
no
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Page 7
Yes (2009)
no
no
Still in progress Fall 2011
no
no
2011-2012
no
III. Student Learning Outcomes Inventory Update

Percentage of courses in your discipline that have CLOs and rubrics developed:_100%_

Percentage of courses in your discipline that have the minimum number of CLOs developed: (1
unit = 1 or more CLO, 2 units = 2 or more CLOs, 3 or more units = 3 or more
CLOs)___33%____ (Astro 30 only; Astro 10/20 have 2 CLOs, and need another).

Date the CLO Assessment schedule was submitted:_2/21/2010

Percentage of courses in your discipline that have had all the CLOs assessed within the past three
years, as per Chabot’s Assessment policy: _50%_
Some sections have not been assessed. Some faculty have the data, but not the time to enter
the results.

Percentage of courses in your discipline that have had all the CLO assessments reflected upon, or
discussed with colleagues, within the past three years__100%

What questions or investigations arose as a result of these reflections or discussions?
1) Does the use of online tutorials and simulations (such as those included in AstroPortal or
Mastering Astronomy) really improve student learning?
2) Are lab students really understanding what accuracy and precision mean in terms of lab
course? What can we do to emphasize their difference and applicability in science?
3) Can we use a 30-question objective test as a common CLO shared by all faculty teaching the
Astro 10 and Astro 20 courses? What can we learn from the results.

What actions has your discipline determined that might be taken as a result of these reflections,
discussions, and insights?
Actions planned:
1) Complete entry of CLOs for Astro 10 & 20, PLOs for the program in general, and reporting on
data already gathered.
2) Investigate shared CLOs in use at other schools showing even better student success, and see what
we can implement at Chabot.
3) Continue to develop CLOs that assess how the improved planetarium AV systems aid in student
learning, retention, and success.
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Page 8

What course-level and programmatic strengths have the assessment reflections revealed?
Strengths revealed:
After implementing the first CLO for Astronomy 30 lab, we reviewed results, modified the
instrument, and added a new lab activity on precision and accuracy (in use for the first time in Spring
2011). We will assess the results of this change this term. This is the first example in our program of
a complete CLO cycle. We hope to use results from Astro 10’s new 30-question CLO online
assessment of student learning this term as well, and review results in 2011-2012.

Percentage of programs within your discipline that have established at least two PLOs, and
mapped appropriate CLOs to them:__0%__

Which of the CWLGs do your discipline’s CLOs address?
Communications and Critical Thinking.

In which if any of the College-wide Learning Goals Faculty Inquiry Groups have discipline
member(s) participated?
None_______________________________________________________________
VII. Academic Learning Support
What kinds of academic learning support does your discipline use or require to help students succeed
(e.g., tutoring, learning assistants, student assistants, peer advisors, lab support, supplemental
instruction, peer-led team learning, peer advisors)? How many hours per semester do you use and/or
how many hours per semester do you need?
As shared earlier, student assistants or part-time lab support is essential for the maintenance of our
astronomy and physics lab courses. With that help, we could much more easily implement new labs
involving digital imaging, new telescopes, radio astronomy, and virtual telescopes.
Chabot College Astronomy Program Review – Year 3 – March 2011 – Scott Hildreth
Page 9