Salvadori measures effectiveness through a six-tiered evaluation system:

1. Independent Assessments – conducted by Youth Studies, Inc.
2. NYC Department of Education Assessment
3. School Teacher and Principal Evaluations
4. Ongoing Observation and Coaching
5. New Educator Onboarding and Mentoring Plan
6. New York University Research Project

Independent Assessments – conducted by Youth Studies, Inc.

Salvadori is committed to positive outcomes for our students and uses independent, third party assessment to measure impact. We have established three primary objectives, measured by increases in students’:

• comprehension
• confidence in their ability
• motivation to pursue STEM education and career choices

Our independent assessment team, Youth Studies Inc., has determined that students who participate in our programs show statistically significant gains in comprehension of math and science concepts, as well as perception of their ability to succeed in STEM-related careers.
Performance results for eight consecutive years show that Salvadori students:

• felt more confident in their math and science ability
• were more motivated to pursue educational/career choices in STEM fields
• improved their knowledge of math, engineering, and architecture concepts
• increased their understanding of the scientific inquiry process

 
Our most recent assessments:

An Evaluation of the Salvadori Center’s STEAM Enrichment Program in NYC Schools (2018-2019)

• Salvadori students demonstrated an increase in their confidence that they can be successful
• Students received an 81.8% on their post-test score measuring confidence in their ability to succeed in math and science
• Students demonstrated an 11.3% increase in their post-test score measuring their motivation to pursue educational and career choices in math and science

An Evaluation of the Salvadori Center’s STEAM Enrichment Program in Scranton and Riverside Schools (2018-2019)

• Students in Scranton District improved their knowledge by 80%
• Students in Riverside District improved their knowledge by 63%
• Salvadori students demonstrated an increase in their understanding of the effects of forces on moving objects
• Students in all schools surveyed increased their test scores by 27.4% after taking Salvadori classes

Independent Assessment Report by Curricula ~ Meta-Analysis (2011-2017)

• Bridges and Building Green participants demonstrated a significant increase in their motivation to pursue educational and career choices in science
• Scranton students scored 78.7% on their post-test assessment of confidence in their ability to succeed in math and science
• Riverside students scored 79.2% on their post-test assessment of confidence in their ability to succeed in math and science

2. NYC Department of Education Assessment

 
The Salvadori Center is an approved vendor for the New York City Department of Education (NYCDOE), Department of Youth and Community Development, and Department of Cultural Affairs, as well as the Scranton and Riverside, Pennsylvania school districts.

Vendor evaluations provided through the NYCDOE’s vendor portal ranks Salvadori “Exceptional” or “Satisfactory” on all questions.
 

3. School Teacher and Principal Evaluations

 
“We immediately noted a tremendous difference in attitude towards mathematics. We began to see wonderful hands-on projects that involved the very skills in which our students were weakest. Perhaps the best symbol of success was brought to our attention by the 9th grade math teacher who asked why her new 9th grade students were so superior to the students who came to her in previous years.”
Monte Joffee, former Principal, The Renaissance Charter School, Queens

“Our math scores were outstanding for our students this year. 94% of our students achieved in performance levels 3 and 4 on the NYS math assessment. We feel that the Salvadori experience helped our students learn and practice important math skills while they were engaged in exciting and fun-filled hands-on activities.”
Principal Valerie Sawinski from JHS 185, Queens

“Not only did my class look forward [to] building various bridges, they learned how to work collaborating and problem solve with their peers.”
Salvadori Starter Teacher, PS 198

“I would absolutely recommend this program to others.”
Salvadori Starter Teacher, PS 179

“I really enjoyed the Salvadori experience and have learned many aspects that I can incorporate in my teaching and professionally. The topics and lessons (especially the materials) are very engaging and allow the students to understand the concepts.”
Salvadori Starter Teacher, PS 192

Principal Survey Findings

Salvadori’s survey of participating principals asked them to indicate their level of agreement with statements across various categories of service. The results were extremely positive:
 
Salvadori Residencies

• My Salvadori program(s) aligned with grade-specific Common Core Math Standards and Mathematical Practices. (score: 8.7 out of 10)
• My Salvadori program(s) aligned with New York State Standards for Math, Science, and Technology. (score: 8.7 out of 10)
• My Salvadori program(s) supported school-wide and district-wide goals as outlined in the Citywide Instructional Expectations and the Quality Review Rubric. (score: 8.3 out of 10)
• I would recommend Salvadori programs to other principals. (score: 9.5 out of 10)

Salvadori Educators

• The Salvadori Educator was knowledgeable about the subject matter. (score: 9.5 out of 10)
• The Salvadori Educator was well-prepared. (score: 9.5 out of 10)

4. Ongoing Observation and Coaching

 
Salvadori has a thoughtful observation and coaching plan conducted by both Senior Educators and the Education Director. Observations are conducted on a frequent basis and include a pre-observation meeting, the observation, and post observation feedback. The observer focuses on collecting and providing feedback to the Educator to ensure high quality teaching of the material using a rubric to capture the information. After feedback is provided, a follow up observation is conducted to measure progress.
 
 

5. New Educator Onboarding and Mentoring Plan

New Educator Onboarding

Our Educators are selected through a competitive application process. Experience with science, math, architecture, art-making, and design, and a passion for project-based learning are the most preferred qualities. While we fully train Educators in the Salvadori approach and current educational pedagogy, teaching experience, specifically with a collaborative project-based approach, is highly desired. Select applicants interview with our Education Director. Successful candidates interview with the Education and Executive Directors. The final candidates must develop and teach a test lesson that embodies Salvadori’s core principals of collaborative, project-based learning to their potential peers ~ current Salvadori Educators.

Once hired, all new Educators undergo an intensive training program. Before being given their own classes, they shadow a variety of seasoned Educators to observe how Salvadori’s programs are implemented. Like traditional student teaching, they assist with the implementation of the programs and eventually teach a session under the guidance of an experienced Salvadori Educator. Once assigned a class of their own, they are assigned a Mentor who observes their classes and guides them to ensure the highest quality program delivery.

Mentoring

Most of our Educators are trained built environment professionals such as architects, interior designers, and urban planners. We provide training in instructional methods to increase their understanding of current pedagogy and educational practices. No new Educator teaches until they have spent a semester shadowing several Salvadori Educators. New Educators observe, learn from, and assist Salvadori’s best and most experienced Educators. In their second semester they begin to teach, still under the mentorship of a Senior Educator. This practice ensures that our schools and students receive the highest quality instruction.

Educators receive a variety of Professional Development (PD): weekly training on curriculum implementation and best practices around classroom management and pedagogy; peer-led Lunch and Learn seminars; subject-matter training from outside artists, architects, and engineers.

6. New York University Research Project

 
The NYU/Salvadori Center collaboration sought to examine the effectiveness of the Salvadori Center’s project-based approach in illustrating the relevance of math and science for students. Specifically, we tested the hypothesis that participation in Salvadori would increase students’ perception that math and science are relevant and useful in everyday life.

• During the weeks 0 through 20 of the academic year, before participation in Salvadori. Results indicated that the average child’s (i.e., regardless of age or gender) perceptions of utility value in math and science significantly declined
• During the weeks 20-28 of the academic year results indicated that, without participation in Salvadori, the average child‘s (i.e., regardless of age or gender) perceptions of utility value in math and science continued to decline significantly through the middle 8 weeks of the academic year
• Perceptions of utility value increased significantly over the course of children’s participation in Salvadori
• During the 12 weeks of the academic year, results indicated that, without participation in Salvadori, the average child’s (i.e., regardless of age or gender) perceptions of utility value in math and science continued to decline significantly from week 28 through to the end of the academic year
• During week 28 through to the end of the academic year, those with participation in Salvadori acquired UV scores that did not decline significantly after Salvadori program completion

Results indicate that although children’s perceptions of utility value in math and science typically decline throughout the school year, participation in Salvadori modules is associated with stronger perceptions of relevance in these subjects. This relationship persists for at least 12 weeks after participation. As greater utility value is associated with more interest in math and science, this program seems beneficial for children’s attitudes toward math and science.