Hi Pavel

Thank you for your comments.

I’m just going to address the first point you raised where you failed to see the link between our KPIs and MSF Australia’s operations, which you highlighted as recruitment and fund-raising.
Allow me to refer you to a quote from the MSF Australia’s president, Rowan Gillies, in the 2006 Annual Report (pp 10):

“MSF Australia strengthened its ability to support operations over the year, with the development of the Project Unit as part of the operational directorate based in Paris, focusing on the quality of paediatric and obstetric care, as well as providing field support in those areas.

This was a functional result of the move over the last few years for MSF Australia to go beyond its current essential tasks of providing high-quality field volunteers, significant financial support and communications back-up for operations.

This occurred in conjunction with a development in the relationship that MSF Australia has built with the various Operational Centres of MSF. We are now one of four sections (France, USA, Japan and Australia) who have responsibility for the operational direction based in Paris.”

http://www.msf.org.au/docs/reports/msfa_annual_report_06.pdf

This provided us with the evidence that MSF Australia’s efforts have, and are continuing to, extend to an operational level. With this in mind we have identified the all of our measures, especially the cost effectiveness KPIs, in the context this future trend, as per the innovation criteria.

Hi Pavel,

Thank you for taking the time to critique our submission. I will attempt to answer the questions you raised about awareness.

I apologise if the video was unclear in expressing how awareness will be measured but as per the written document, awareness should be measured by comparing the number of crisis situations MSF attends to, to a list compiled with the co-operation of all other NGOs. This list should include all crises identified by each organisation as a result of all their different objectives and the greater the diversification, the more superior the list will be.

It is this list that MSF can then use to compare their ability to contribute aid to the overall/ worldwide situation and at the same time perhaps make them aware of other crisis situations which they could potentially become involved in.

So the awareness KPI identified by our group is more about MSF comparing their work to the world situation rather than directly comparing it to the work of another organisation.

Hope that answers your queries and thanks again for your comments.

Jess

Hi Shuangchen,

Thank you for your comments about our submission. As the person who wrote the awareness section of the report, i'll nominate to answer your question.

I agree whole heartedly that donators will not necessarily do so only via internet. That is why as per our word document, i have outlined the various other methods that can be used to not only the level of donations but also the overall public awareness of MSF.

As per for word report, all other contact forms should be considered- such as telephone, fax, postal letters and the number of volunteers signing on to be part of the organisation. In addition, in the report, i've also highlighted the importance of regular surveys to gage public knowledge and awareness of the existence of MSF and the work they do.

I hope that answers your question. Thanks again for taking the time to critique and comment

Jess

Hi guys,

Your report is truly a pleasure to read. You have taken great care in researching best practice and finding data to back your ideas. The multimedia presentation is perfect. Five stars.

My comments below are related to your pdf report.

I especially liked your Justification sections – thorough, realistic and any caveats noted.

You developed an interesting spin on awareness in your ‘Awareness of humanitarian crises’ measure. While I do not believe that the MSF are unaware of any current/recent crises themselves (since they are the ones who compile lists of the most underreported stories), this indicator is a very clever one for the MSF Australia’s marketing and annual reporting purposes. It sums up nicely the extent of their humanitarian involvement and motivates the donors at the same time.

A brave attempt to tackle performance measurement issues in the Ethics area. An ethics scorecard is a fresh idea, but who will assign scores? I think it should be done by an independent third party. Subjectivity can be mitigated through standardisation and peer review. As an example, Monash Sustainability Enterprises (MSE) currently provides rankings and assessments of companies’ ethical behaviour for use in socially responsible investing.

The only overall deficiency I can think of is that your report is not targeted to MSF Australia. Your KPIs can be used by any partner section within the Movement who has operational responsibility. On the flip side, a universal application of the same indicators would be great for the MSF International as it makes inter-section benchmarking and comparisons possible.

Hello HSC,

I can obviously see that you guys have a funny set of priorities. Wonderful report, with just a few questions.


Awareness: I don’t seem to understand how the number of ‘crises’ attended to over the number of ‘crises’ actually happening is any idea of awareness, at least to the Australian public. You do hint some good ideas in the Role of MSF dot point, but the others seem to be coming from the wrong angle. The ‘awareness of MSF Australia’s work in other countries’, seems slightly misguided. Your mathematical equations is essentially ‘how much of the worlds crisis’s MSF are covering’ which measures MSF’s awareness towards international crisis’s, not of their own awareness by others in the community. Although if you think one will lead to the other, then you’ll be saying the tail leads the dog.

Hi Carlo,

Thank you for your taking the time to review our submission. I will aim to answer your questions as best I can.

Q= “I don’t seem to understand how the number of ‘crises’ attended to over the number of ‘crises’ actually happening is any idea of awareness, at least to the Australian public.”

A= The method of comparing the number of crisis situations MSF attends to in relation to the total global number is not actually directly intended for the Australian public. The purpose of the comparison list is more for MSF’s use so they can evaluate whether the work they are currently doing is enough, or whether they could expand their operations and support.

Therefore, using this method allows MSF to increase their own awareness, which will then be forwarded on to the public through the regular updates MSF gives out about the work they do.

It is only then at this point that the public’s awareness is directly measured. I.e. – using the number of website visits, emails, faxes, postal letters, surveys and volunteers.

In the report, the awareness indicator was written in the format so that the next dot point is a flow on from the previous. I feel this is where your confusion may have arisen as perhaps you’ve interpreted each dot point as a separate method of measuring direct public awareness. If this is so, I apologise for the inconvenience this has caused you whilst reading the report.

Hopefully this will have cleared up any of your concerns. I’d just like to ask you to clarify what you meant by “funny set of priorities?”

Regards,

Jessica

Dear Judges,

Thank you for reviewing our submission. We will address your questions in two parts.

1. With regards to your DALY KPI, what do you think would be an appropriate time frame to measure this KPI in?

HSC recommend that an appropriate timeframe to measure the DALY KPI is on an annual basis. This would allow findings to be included in the Annual Report and therefore be able to be used by key stakeholders in analysing performance over time.

It is important to note that the DALY KPI is a flexible measure. As in our report, we alluded to the fact that we could measure the DALYs saved subsequent to a specific intervention (e.g. immunisation program) and compare it to the cost of that intervention to establish cost effectiveness. This could also be included in the Annual Report to provide more disclosure and therefore be able to compare cost-effectiveness between intervention programs, adding to the transparency of MSF.

As our recommendation above, we believe that measuring cost-effectiveness on an annual basis would be desirable. This would require an assessment of DALYs saved from all the interventions for that year, to be then compared with the funding input for that same period in order to calculate cost-effectiveness.

It is important to realise then there are certain measurement issues raised by this method. Firstly, the fact that intervention programs may run from one year to the next, and secondly is that some interventions do not save DALYs immediately, especially prevention strategies such as immunisation. In any case, as long as the methods are consistent (such as discounting future DALYs to the present value) and disclosed within the Annual Report, we believe that these issues can be controlled.

2. How does this relate to the Australian operations of Médecins Sans Frontières?

HSC believe that the generic applicaton of the DALY KPI is its strength. Hence, we believe that through the use of the DALY KPI, MSFA would be able to compare the cost effectiveness of the operations it contributes to and therefore allows tracking of efficiency (cost-effectiveness) over the longer term.

While we recommended DALY KPI be measured on an annual basis, the indicator is flexible and therefore allows MSFA to include short, medium and long term evaluations. This would be to the advantage of MSFA, as no one can predict duration of intervention programs when a crisis occurs, especially if it is unexpected.

Recently MSFA has begun directing its own operations. Given Australia’s large resource base – both monetary and professional, developed country status and social conscience, We believe this operational focus is set to increase in future years. For this reason we put forward cost per DALY averted as a key performance indicator of operational funding which attracts approximately 80% of the budget. We believe it is imperative for MSFA to have this strong evidence base for cost-effectiveness analysis of its interventions, in order to achieve equity and efficiency in the allocation of its resources.

As MSFA is part of MSF International, comparisons can be made easier if the same data was collected across all intervention programs. Hence, the DALY KPI could be used for ranking purposes of the different MSF organisations worldwide, especially with regards to funds and donations allocation.

Application and reporting of DALY measures by MSFA is also important for accountability and transparency. Reporting the DALY measures for operations supported by Australian donations allows donors to clearly see how their funds are spend and whether this represents an efficient use of funds.

Hopefully, that helps and highlights the flexibility and strength of the DALY KPI.

Please feel free to get in touch if there are any other matters.

Kind regards,
HSC

Hello all!

I'd like to acknowledge the impressive clarity of your report... Congrats with that! All the same, I've some difficulty understanding a few of its aspects. In your synopsis, a KPI entitled: 'disclosure of sources of funding' confuses identification of MSF's various funding sources with the actual sources of funding. Also, I dont understand your explanation of how cost per daly is applicable to MSF Australia. I felt the explanation a bit roundabout, that's all:) good luck to yer!

Hi Sinead,

The idea of of the "disclosures of sources of funding" vs "actual sources of funding" is that the aim of this KPI is to generate independence, accountability and transparency.

We believe this is achieved by "disclosing" MSF sources of funding to the public - not simply identifying it for internal use.

With regard to the question on DALYs: fair comment, we've tried to clarify it through the comments section. Its addressed the response to the Judges by Joel Seah, which is the post above yours.

Thanks for taking the time to review our work.

Sam Gray

Hi Olga,

Thanks alot for taking time to read and comment on our submission.

I agree with you that the ethics score should be given by an unrelated third party so as to maintain the independence and ethics of MSFA.

With regards to the report not aiming at MSFA. We believe our report (KPI and measurements) are aimed towards MSFA's future and how the organisation is continually changing and evolving to be a bigger presence globally.

Thanks for your constructive comments.

Much appreciated...

Cheers
Joel

Hi all,

On behalf of HSC i'd like to thank everyone who took the time to read our group's submission as well as post constructive comments and questions.

This was the first year we entered and it has been a highly motivating experience.

All the best to the other teams!