Call Us (949) 396-6053

OUR IMPACT

What a local community-based organization learned from the results of an impact evaluation

FOREVER FOOTPRINTS

Forever Footprints is an organization focused on families who have experienced infant or pregnancy loss or infertility. The organization provides direct mental health support services, educates the medical community to improve the quality of care and response, and offer opportunities for remembrance to help families find their path to healing. 

The Mark conducted an impact evaluation for Forever Footprints. Our external assessment examined how programs were being implemented, who was being reached, how participants were being affected by programming, and how the programs were doing in achieving the overall goals of Forever Footprints, as outlined in their strategic plan.

We began by working with the client to collaboratively develop a logic model for the organization. We then:

  • Identified all existing data sources that would allow us to examine the different programs and initiatives.
  • Assessed the impact of the organization by examining the identified data sources (dating back to 2015), which included spreadsheets tracking participation, donations, and registration, and surveys developed and administered by Forever Footprints. 
  • Analyzed the data using descriptive and inferential statistics and reviewed all results with the Executive Director to ensure they were correctly interpreted and framed with the appropriate context.

Submitted a final report that included a summary of results, key findings, and detailed recommendations. 

The Executive Director was able to demonstrate the impact of the work Forever Footprints is doing to key stakeholders such as the Board, funders, and program staff.

The recommendations made by The Mark were brought to the Board to modify programs, improve internal data collection and monitoring, and establish and foster the necessary partnerships to improve the work. 

The organization was also able to demonstrate which outcomes it had achieved, and which ones were lacking but could be changed based upon different decisions so that they could be more effective. The results of the findings could also be used to justify or articulate specific reasons for why additional funding was needed or warranted.

UNIVERSITY OF CALIFORNIA, IRVINE

How University of California, Irvine received multiple three-year grants

The Water-PIRE project was a project for the University of California, Irvine (UCI) Partnerships for International Research and Education (PIRE). This project was a five-year international scientific collaborative project in the area of urban water systems, using transdisciplinary approaches to tackle urban water issues, and was funded by the National Science Foundation (NSF).

One of the goals of the project was to increase knowledge and understanding of sustainable urban water systems, and in the process equip a new generation of engineers, natural, physical, and social scientists, policymakers, and educators with multi-disciplinary skills and sensitivities.

The project also aimed to accelerate education and training in the area of urban water sustainability and share this knowledge about sustainability options with U.S. middle-school and high-school students, undergraduate STEM majors, graduate students, post-doctoral researchers, and practitioners.

Lastly, the Water-PIRE project wanted to establish new partnerships between university researchers, non-university researchers, and urban water managers to improve urban water sustainability research and application.

For this multi-site international project, we conducted a formative evaluation to improve program implementation and a summative evaluation to demonstrate progress made towards project goals and examine the broader impacts.

 

The Mark also:

  • Used a mixed-methods approach and collected quantitative and qualitative data.
  • Conducted an annual survey with all project participants as well as activity surveys with targeted participants.
  • Conducted focus groups and interviews and did onsite observations of grades K-12 and the Undergraduate PIRE program (UPP) Down Under.
  • Conducted descriptive and inferential statistics on all quantitative data and coded all qualitative data for themes.

The project leaders were able to leverage our evaluation results to apply for subsequent funding and went on to receive a three-year NSF Research Experience for Teachers (RET) grant and a three-year grant from the UC system, a Multicampus Research Programs and Initiatives (MRPI) award.

The team has gone on to also win an NSF Engineering and Research Center (ERC) planning grant and will apply for an ERC grant. 

UCI used our evaluation results to improve their programming yearly and inform their key stakeholders about project progress and impact. Each year we offered information on key results and made recommendations for program improvements and growth.

After the project, we were able to participate in the external advisory board meeting and provide a third-party review of the program and all progress achieved during the five years.

BENIN PINEAPPLES

The Pineapple Project: An international nonprofit case study

The Pineapple Processing for Export (PINEX) project in Benin was a five-year project of PfD whose goal was to strengthen the pineapple value chain at all levels in Benin, Africa while working with all key pineapple stakeholders (pineapple producers, processors, exporters, government, financial institutions, civil society organizations, etc.).

Partners for Development (PfD) works in partnership with local and international groups to improve the quality of life for vulnerable people in underserved communities. The focus of PfD’s work includes agricultural development, economic empowerment, and healthy communities.

This project comprehensively addressed challenges in the pineapple sector of Benin by including training and capacity building for producers and processors, increasing access to finance and inputs, and growing infrastructure, market information, and brand recognition.

The Mark conducted a mid-term evaluation of the project to review and assess PINEX’s implementation progress, assess if targeted beneficiaries had been receiving services as planned, and to what extent PINEX activities have been effective, efficient, relevant, impactful, sustainable, and on track to meet the project’s stated goals and objectives.

We began by working with PfD Directors to collaboratively develop an evaluation plan for the project. We then identified and reviewed all existing data sources that would allow us to learn the program implementation and develop appropriate evaluation tools. Together with the client, we developed a producer survey and a focus group protocol, created processor and exporter interview questions, and constructed other key stakeholder interview questions.

Sixteen surveyors conducted the field work in Benin within two weeks and collected survey data from 768 producers.  In-depth focus group discussion data was collected from four focus groups of producers with diverse characteristics. Twelve interviews with processors and exporters and ten interviews with other key stakeholders were completed.

After the field work, we analyzed the data and triangulated results from multiple data sources to assess project effectiveness, efficiency, impact, relevance, and sustainability. We also identified challenges of current project implementation and provided actional recommendations for PfD to improve their work in the next two project years.

A full report was provided to PfD so that PfD could share it with key stakeholders and their funder. Throughout the data analysis and report writing stage, we had several detailed discussions with PfD to talk about the key findings and recommendations and answer any questions that they had so that PfD could fully understand the key findings and recommendations. The whole evaluation period was three months for this project.

PfD submitted our evaluation report to the funder and demonstrated what worked well and what had not worked well in the first three project years. PfD also disseminated the key findings and recommendations to key stakeholders to show the implementation progress of the project and share the success and challenges of the project.

According to the Director of PfD in the Benin office, it previously took a lot of time or was very hard for PfD to demonstrate the impact with such massive and fragmented information from everywhere. The report helped them to gather, analyze, and centralize all kinds of information in one place and made it much easier for them to use the information to learn about the project implementation and demonstrate impacts systematically.

PfD also learned the obstacles and areas that needed to be improved through the evaluation and has made plans to improve the project implementation and maximize program impacts so that they were successfully positioned to receive additional funding.

Center Club becomes an agent for change in Orange County

CENTER CLUB OF ORANGE COUNTY

Center Club Orange County hired The Mark to manage, evaluate, and present the nonprofit charity submissions for selecting their annual Center Club Cares partnership and $10K grant recipient.   

Center Club Cares is the philanthropic arm of Center Club Orange County, which is a business social club that fosters networking and business relationships as well as direct involvement in the community.  As part of its mission, Center Club Cares focuses on community development and workforce or leadership development. 

For their annual grant, Center Club Cares considers nonprofit community development programs that are focused on bringing about change for social good and improving the quality of life to citizens in the Orange County community. Additionally, they will consider workforce or leadership development programs that provide job training opportunities, employment services, support for job retention/stability, advancement to better jobs, technical training, and strengthen the Orange County workforce.

The Mark helped Center Club Cares to assess its process for evaluating and determining which nonprofit to sponsor and provide a financial gift on an annual basis. The Mark helped revise the application process and assist the committee in evaluating all applicants by considering different key attributes that they may not have otherwise considered. We also provided recommendations to the committee on how to improve the process in the subsequent year and more importantly demonstrate how to apply some better practices for choosing a recipient as a donor.

Presentations on applicants were prepared three times and presented to the committee to evaluate the selection of the grantee. 

As part of being involved in the process, The Mark was able to advise Center Club Cares on how they can be more effective in its donation process and help them work more directly with the non-profits to demonstrate outcomes vs. outputs better and establish greater transparency and accountability between themselves and the recipient. 

The Mark also was able to advise Center Club Cares on how to improve the giving process so that as a Funder, the committee can begin to understand better the significant role evaluation can have on non-profit organizations and help the latter be more effective organizations that focus more on outcomes and not outputs and efficiently use resources. 

This has helped Center Club Cares be recognized as an agent for change among donors/funders and increase its credibility as a group for being a leader that influences change in terms of how money is donated to non-profit organizations.

US DEPT. OF DEFENSE &
UNIV. OF SOUTHERN CALIFORNIA

Helping the U.S. Department of Defense and USC promote development in California

The Mark was hired to evaluate the AMP SoCal Strengthening Competitiveness Program (SCP) to provide the program managers at the USC Price Center for Economic Development with insights into the effectiveness of the program.

The U.S. Department of Defense funded the AMP SoCal project (Advanced Manufacturing Partnership), which was implemented by USC, to promote economic development. The goal of the program is to strengthen firms’ competitiveness in non‐defense markets to maintain expertise and jobs in the region.

AMP SoCal runs programs like the Red Carpet program that are aimed at increasing the participation in AMP SoCal growth activities by firms that have been adversely affected by reduced Department of Defense (DoD) procurement and contracting.

Another program, the Managed Career Pipeline program, focuses on providing training to unemployed and other eligible candidates for entry‐level technician positions or workers in need of upgraded skills.

The Mark used archival document analysis and interviews to evaluate AMP SoCal sub-components. The Mark received program documents (like monthly performance reports and scopes of work from the USC Center for Economic Development) along with contact information for the sub-component project leaders.

The Mark conducted interviews with the project leaders to kick off the evaluation, entered information referenced from the interview into a Data Tracking Form, and followed up with project leaders to obtain any further necessary data. The Mark used Dedoose to conduct qualitative analysis and Google Sheets to create datasets, and R, SPSS, or Stata to run quantitative analyses.

AMP SoCal gained insights into how effective they were in meeting their goals and objectives of job creation, workforce training, and business development in the aerospace and defense industries. They were able to use the information The Mark provided to improve programming and demonstrate the impact of their work to the Department of Defense. The Mark also provided AMP SoCal guidance on priority areas for future evaluation work.

×