Download Assessment Outcomes Report 2008/09

Computer Information Systems
Assessment Outcomes Report 2008/09
Contents
Computer Information Systems ................................................................................................................ 1
Assessment Outcomes Report 2008/09 ................................................................................................... 1
Contents .................................................................................................................................................... 1
Summary of Assessments ......................................................................................................................... 2
1. Program Review – Every 4 Years ........................................................................................................... 3
2. Department Assessment ....................................................................................................................... 5
2.1 Learning Goal #1. Develop professional Written and Oral Communication Skills. (CIS 328
assignments, CIS 466) ........................................................................................................................... 5
2.2 Learning Goal #2. Develop broad technical knowledge in core CIS areas. (CIS 328) ..................... 9
2.3 Learning Goal #3. Develop analytical abilities in problem solving. (CIS 304) ............................... 10
2.4 Learning Goal #4. Develop Teamwork Skills and Knowledge of Project Management and
Development Processes (CIS 466) – Final Presentation ..................................................................... 11
2.5 Learning Goal #5. Develop Integration Skills between technologies and different disciplines. (CIS
466) ..................................................................................................................................................... 12
2.6 Learning Goal #6. Develop professional ability to work with a client. (CIS 466) .......................... 13
College Assessment................................................................................................................................. 14
Summary ................................................................................................................................................. 16
Suggestions for Future Improvements ............................................................................................... 17
1
Summary of Assessments
1. Develop professional Written and Oral Communication Skills. (CIS 328 assignments, CIS
466)
2. Develop broad technical knowledge in core CIS areas. (CIS 328 pretest)
3. Develop analytical abilities in problem solving. (CIS 304)
4. Develop Teamwork Skills and Knowledge of Project Management and Development
Processes (CIS 466)
5. Develop Integration Skills between technologies and different disciplines. (CIS 466)
6. Develop professional ability to work with a client. (CIS 466)
Program Review
Departmental Assessment
College Assessment (CIS 310)
Every 4 Years (Summer Olympics) (Fall Term)
 Look at Model Curriculum
 Examine other Programs
 Hold an advisory board meeting to get
feedback from industry experts.
Every year (Spring )
 Intermediate Java - CIS 304
 CIS Careers - CIS 328
 Senior Project – CIS 466
Each Term
 Administer 10 questions on information
Security at 310 Final Exams.
2
1. Program Review – Every 4 Years
AY 07/08 – Examine model curriculum

Drew Hwang collected information on all CIS programs in the United States. Results showed the
CIs program to be similar to many nationally. The CIS program typically has more depth and
offers more hands-on, technical courses.

Our program was found to be consistent with the model curriculum. Our strength was in
offering advanced electives that other programs didn’t have.

Ralph Westfall examined the model curriculum from ABET and AIS.
AY 08/09 – Industry advisory board (IAB) meeting outcomes


Problems at the Time
o Getting more majors – enrollment was flat at about 470.
o Attracting more women to the major
o Not enough faculty
Advisory Board Suggestions
o To attract more majors - Develop an introductory course to show students the big
picture, give them an overview of what CIS is and how the different courses will relate
to the Systems Development Life Cycle.
o ‘Decide who you’re going to be.’ – Due to the lack of faculty, pick an area for the
department to specialize in so the faculty are not spread too thin.
o Topics that IAB liked
 Project Management
 Mobile Devices
 Advanced Web Topics such as Silverlight, Drupal
 Open Source Solutions
3
Actions 08/09
Attracting More Majors
Several PR documents were developed and the Web site was expanded to
tell potential students what CIS is. Outreach events promoted the program.
Information from the change of major students shows that ECE and CS is
where most of our CP majors transfer from. This is by far, most of our
students. About 100 majors a year come from ‘change of majors’.
Students taking 310 don’t change majors to CIS. The recruiting for new
majors on campus could take place in 101.
During 08/09, 54 new freshman entered CIS. CIS had only 20 new freshman
that attended orientation. Transfers from Community Colleges has been
limited to Fall only.
The 08/09 number of majors CIS is 467. The prior year it was 469.
Attracting More Women
to CIS
A mentoring program was set up by Louise Soe. This is publicized at the
orientation sessions for new students.
A Women’s Panel was held at the CIS 40th reunion.
Discussion about this topic has raised awareness in the department. Faculty
are great at promoting the women we do have.
Not enough faculty
The budget crisis has kept us from recruiting new faculty. We have not
eliminated any advanced electives yet. That will probably take place in
09/10.
4
2. Department Assessment
Learning Goals
1. Develop professional Written and Oral Communication Skills. (CIS 328 assignments, CIS
466)
2. Develop broad technical knowledge in core CIS areas. (CIS 328 pretest)
3. Develop analytical abilities in problem solving. (CIS 304)
4. Develop Teamwork Skills and Knowledge of Project Management and Development
Processes (CIS 466)
5. Develop Integration Skills between technologies and different disciplines. (CIS 466)
6. Develop professional ability to work with a client. (CIS 466)
2.1 Learning Goal #1. Develop professional Written and Oral Communication Skills. (CIS 328
assignments, CIS 466)
CIS 328 – Resume and Cover Letter – Apply this Rubric to Measure Written Communication Skills
Great
Good
Average
Unacceptable
Grammar and
Format
Clear Objective
Student has
technical Skills
that are clearly
articulated
Cover Letter is
professional and
to the point
This rubric wasn’t actually used. See the table below for data collected from CIS 328. Grades for the
cover letters suggest that students lack writing skills. Of 17 recorded grades, 4 students got 5 or 10 and
3 students got 6 of 10. 5 and 6 indicate the number of grammatical errors found on the cover letter.
For a short letter, that is often modified from a template, this is remarkably poor.
5
Winter 09-CIS 328
Student
Total
Points
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Average
Cover
letter
Instructor: Carlos Navarrete
Resume
10
9
6
6
8
5
9
5
8
9
5
5
9
8
6
7
10
8
10
9
7
10
8
7
10
9
9
9
7
8
9
10
8
10
10
10
10
8
6.473684 8.842105
Grading system:
Total points: 10 points
One grammar or spelling mistake: -1 point
Grade of five reflects more than five grammar or spelling mistakes
6
CIS 466 – Final Presentation to Client – Apply this Rubric to Measure Written and Oral Communication
Skills
Great
Students have
clear agenda and
stick to it.
Student handouts
are professional
and clear.
Presentation is
clear, relevant and
interesting.
Each team
member shares in
the presentation
of business
solution.
Team
communicates
technical topics in
a way that a client
can understand.
Students
communicate with
technology.
(shared document
group, email, IM,
Blackberry…etc.)
Good
Phoenix
Therion
LLC
Emanon
LLC
Emanon
Phoenix
Therion
Emanon
Phoenix
Therion
LLC
Emanon
Phoenix (3)
Therion
Emanon
Average
Unacceptable
Therion (started
15 minutes late)
Phoenix (no
agenda- Fabulous
Presentation.
LLC (too technical
for client to
understand)
Phoenix (3)
LCC
Phoenix
(Sharepoint)
Therion (Drupal)
LLC (Ajax, PhP,
Drupal, Google
Groups)
Emanon
Four Teams were assessed –
Phoenix – 6 students, Client Baxter Healthcare, Sharepoint Server Project

No agenda was passed out. It really wasn’t a big deal. The team used Prezi instead of
PowerPoint. The Prezi documents were fabulous.

Students met client once in Woodland Hills then teleconferenced. Client raved about their
ability to communicate and gave $1,000 to the department.
7
Therion – 6 students, Client CP Gail Waters – Entrepreneurship Portal/Website (*service learning)

Great presentation. Very relaxed and clear. Very knowledgeable students.

Sean Collins knew lots of different technologies and did a trade before starting the project. He
gave a great overview of what Drupal was. Excellent in explaining technology to client.
Serious Business LLC – 6 students, Client Sandra Senft & Fred Gallegos, IT Audit and assurance Web site.

Data in demo was dumb. Use data that is more reflective of client needs.

LBD and SDLC mentioned and used.

Used Drupal, then scrapped it and hand coded the site.
Emanon – Search Engine Optimization, 5 students, Client Drew Hwang, Optimize Google Search Engine

This is too small and easy a project for 5 students. However, students created Web sites and ran
experiments on them. Too canned a project with a known outcome. No real client interaction.
Discussion
The Senior Project Presentations were exceptional. Louise had them practice prior to their final meeting
with the client and it was clear that the students were well spoken and technically knowledgeable. All
teams appeared to share the work among team mates.
8
2.2 Learning Goal #2. Develop broad technical knowledge in core CIS areas. (CIS 328)
Administer an assessment test of the core as the 328 final exam. (I did this because people can take DB
concurrently with 328).
Questions:





What are the 7 layers of the OSI model?
Write a small java program that types ‘hello world’ to the screen 7 times.
Draw a class diagram for an ATM machine.
List two things that can make a Web site more accessible.
Write an essay regarding your future IS career. Please include a title and be sure to
observe formal writing guidelines.
Assessment test W09-CIS328
Question
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Average
Telecom
Program
SA&D
Web *
Writing**
10
10
2
5
8
5
6
6
5
4
2
7
8
5
4
10
10
10
10
10 Person has career.
0
2
4
7
7
0
5
4
5
7
7.5
6
4
10
8
10
10
7
5
10
8
4
4
5
6
10
8
10
5
7
2
8
10
5
8
wrong diagram on
10
10
0
5
10 SA&D
wrong diagram on
0
0
0
5
10 SA&D
ERD diagram - not
10
10
0
5
9 class
wrong diagram on
8
0
0
5
7 SA&D
4
4
10
5
9
4
10
10
5
10
8
10
8
5
9
4
10
0
5
10
6
10
4
5
8
6
5
0
10
8
6.225
7.25
5.05
6.1
8.45
9
* 5 indicates an intelligent answer but, not accessibility as it is formally defined
** Really low scores are ESL
Discussion
The test was a ‘pop quiz’, students didn’t have preparation before. The purpose was to see what they
retained.
 Question 1 – OSI Model - A better question might be ‘What is the OSI model and why is it
useful?’ We want to know if they understand the model, not necessarily that they remember
the names of the layers.
 Question 2 – Program - Most students could put together a simple program, some very well.
 Question 3 – Class Diagram - A few students drew sequence diagrams. This indicates that they
know what they are doing but, might not know the name of the particular diagram. More
students drew great diagrams than poor.
 Question 4 – Web – Students gave great answers but, more towards how to get people to see
the site. Accessibility means that disabled people can still work the site. Most students didn’t
understand that accessibility is a term that means this specifically.
 Question 5 – Writing – Low scores were from students who clearly had English as a second
language. They’ll never write a great essay prior to graduation. We need to measure that they
have a way to improve their writing electronically or with review. On a pop-quiz, there was no
way for them to do this.
2.3 Learning Goal #3. Develop analytical abilities in problem solving. (CIS 304)
Not done.
10
2.4 Learning Goal #4. Develop Teamwork Skills and Knowledge of Project Management and
Development Processes (CIS 466) – Final Presentation
Great
Good
Average
Unacceptable
Professionalism
Phoenix
Therion
SBLLC
Emanon
Organization
Phoenix
Therion
SBLLC
Emanon
Clarity
Phoenix
Therion
SBLLC
Emanon
Shared Workload
Phoenix
Therion
SBLLC
Emanon
Clear Process
Phoenix
Defined and
Therion
Followed
SBLLC
Emanon
Ability to Deal
Phoenix
with Problems
Therion
SBLLC
Emanon
Client Interaction
Phoenix
Emanon
Therion
SBLLC
Discussion
See pages 6-7 for project descriptions.
The team that did Search Engine Optimization didn’t have as rigorous a client experience as the teams
that had external clients and larger projects.
11
2.5 Learning Goal #5. Develop Integration Skills between technologies and different
disciplines. (CIS 466)
Great
Good
Average
Unacceptable
N/A
Web
Phoenix
Emanon
Development
Therion
SBLLC
Telecom
Phoenix
Therion
SBLLC
(external
Emanon
host)
Database
Phoenix
Emanon
Therion (CMS)
SBLLC
Programming
Phoenix
Emanon
Therion
SBLLC
Other tech skills Phoenix
SBLLC Drupal,
Emanon
Prezi/Sharepoint PHP, Ajax
Therion PHP &
Drupal
Project is Secure Phoenix
Therion (left
Emanon
to Drupal but,
pretty robust)
SBLLC
Project is
Phoenix
Emanon
Accessible
Therion
SBLLC
Project is
Phoenix
Therion
Emanon
Maintainable
(doubt client
can do Drupal)
SBLLC
Client can use
Phoenix
Emanon
product
Therion
SBLLC
Discussion -See pages 6-7 for project descriptions.
Therion – Entrepreneurship Portal with Gail Waters. Great presentation. Very relaxed and clear. Very
knowledgeable students. One knew a lot about several types of technologies and id a trade before
starting the project. He was expert at explaining Drupal. Team used Drupal, Flash, Hosted at Host
Gator, Swish. Presentation included test and accessibility. Took original pictures for content on Web.
Asked about security and load testing. Load testing was done. Security was only access control. They
did try to upload virus exectules and the Drupal firewall stopped it.
12
2.6 Learning Goal #6. Develop professional ability to work with a client. (CIS 466)
See 1 and 4.
Discussion
See pages 6-7 for project descriptions.

It might be better to collect the client ratings for the students on this learning goal instead of
observing the presentation.
13
College Assessment
CIS 310 – Information Security



Administer Assessment Questions during Final Exam all year. (AY 08/09)
Give results to the College of Business for the ALC to examine. (TBD)
Discuss Results at Faculty Meeting. (TBD)
CIS 310 Assessment Questions
1.
a)
b)
c)
d)
Which of the following is NOT an employee threat to information security?
destroying data
working from home **
sharing a password
carrying sensitive data outside of the firm.
2.
a)
b)
c)
d)
What is phishing?
It tracks keys that people click on their computer.
It is malicious computer programming.
It is sending fake email to get information from unsuspecting users.
It is harmless junk email.
3.
________ is the process that confirms the identity of the remote user who is attempting to
access information from the Web server.
a)
Authentication
b)
Identification
c)
Confirmation
d)
Registration
4.
a)
b)
c)
d)
VPNs use a technology called:
channeling.
tunneling.
mediating.
None of these
5.
Safeguard technology limits access to computer and physical areas. Which of the following is a
safeguard technology?
a)
Passwords
b)
Smart Cards
14
c)
d)
Biometrics
All of the above are safeguard technologies
6.
Firewalls can prevent unwanted access to a firm’s corporate computing network or a home user.
What is a firewall?
a)
software
b)
hardware
c)
software, hardware or a combination of both
d)
none of the above
7.
________ is software that runs on your computer, collecting information which is then
transmitted to another party.
a)
b)
c)
d)
Cookies
Spyware
Spam
Spim
8.
a)
b)
c)
d)
Software piracy is a(n):
ethical issue.
crime.
economic problem.
All of these
9.
a)
b)
c)
d)
Identify which password below is most secure.
password
ABCDEF
A61bcm2
fluffy.
10.
a)
b)
c)
d)
Which is NOT a threat to information system security?
accidents and natural disasters
viruses
employees and consultants
all of the above are potential threats to information security
15
Summary
External Review
The Industry Advisory Board gave feedback that resulted in the development of a new course to
encourage new majors and give a more concrete overview of the role of CIS in organizations. The
course will be reviewed by the Advisory Board in Winter 10 and will be implemented in Fall 10.
Learning Goals
1. Develop professional written and oral communication skills. Students still need work on
writing, earlier in the program. The new course and its earlier delivery in the CIS core should
help students get an early start on this effort and in presentation skills. Students with ESL,
should be aware of how to get help before they turn things in.
Senior Project Presentations were near perfect. Having the student’s rehearse helped all of
them.
2. Develop broad technical knowledge in core CIS areas. (CIS 328 pretest)
The pretest overall was average. Low score were due to students not knowing the correct term
for specific things, though they appeared to be knowledgeable (writing the wrong diagram (class
vs. sequence vs. ERD), giving a very knowledgeable wrong answer (accessibility).) Testing
writing at this point seems ineffective. If a student is a native speaker, they score well. If they
are not, they score poorly. Giving a pop-test on something they cannot possibly accomplish
doesn’t really tell us anything.
3. Develop analytical abilities in problem solving. (CIS 304)
Not done. This may be a better place for the pre-test in that they will have had database and
more exposure to SA&D in practice.
4. Develop Teamwork Skills and Knowledge of Project Management and Development Processes
(CIS 466)
Students were exceptional.
5. Develop Integration Skills between technologies and different disciplines. (CIS 466)
Most projects integrated several key core areas.
6. Develop professional ability to work with a client. (CIS 466)
In house clients may not give as great a client/interaction experience. Using the client feedback
form, instead of observation, will be a better measure of their ability with the client.
16
Suggestions for Future Improvements
1. See if the new 4 unit course does what it was supposed to.
2. The assessment plan will have to be re-written to accommodate the new structure of the
curriculum.
3. Measure a few things tertiary to curriculum that will improve CIS. For example, something to do
with student clubs, integration of ‘Learn-By-Doing’ in CIS courses, or evaluation by employers.
17