Your idea of measuring cost per DALY was very good, but how would can you justify using it if that amount of data is rarely available?
Other than that I was really impressed with both the layout and the content of your report. I was under the impression however that written reports were limited to 5 pages, or does this requirement no longer apply this round?

Hi Eleanor. Thanks for your feedback.

We took the entry criteria as a guide to our report, but found it a bit vague in reference to specifics (no mention of font, page margins, font size or line spacing).

So we wrote a 5 page report + 1 page of references, then applied double spacing to make it more readable in PDF online, which took it to 7 pages. I think a few other groups took a similiar approach (Godfathers, Equilibre, Prudent Management)

Just out of interest Eleanor, did the video link work for you?

Just out of interest Eleanor, did the video link work for you?

Hi Guys,

Your presentation is really good, it is easy to follow and highlighted all the seven key performance indicators with an introduction of MSF’s background.

It is also a pleasure to read your report, it is very detailed and contains a lot of information, you guys must have done a lot of research. The layout of the report is also very good and easy to follow.

The two KPI’s for human resources are great, in our report we had similar ideas compared with yours, but we did not have that much information and detailed analysis. In our report, we focused more on MSF Australia rather than MSF International . You mentioned other possible KPI measurements for HR such as MSF volunteer satisfaction (average) % vs. Industry average. We think that this is a great way of measuring the volunteer satisfaction with the industry bench mark. However, we believe this is more suitable for MSF International rather than MSF Australia , as Australia is a non-operational sector and it may not have control over their staff and it may be a bit hard to compare with the industry average (as the Red Cross and the Salvation Army may be a relatively ‘active unit’ rather than non-operational unit) .

Again, the first two KPI’s in your report are great, but we think it is more suitable for MSF International rather than MSF Australia. As MSF Australia is a non-operational sector, its aims are to collect donations and recruit volunteers. The cost per DALY adverted is great for measuring the cost effectiveness of the organisation as a whole but it may not be suitable for MSF Australia. Also, our main concern is where do you find the data to complete the formula? As MSF is a non-profit organisation, they may not want to spend a lot of money on this area (we understand that the more money MSF is going to spend, the less people are going to be assisted). Perhaps it may be a good idea to do this every three or five years as a whole organisation.

The second KPI listed on your report is the cost per clinical output and the fifth Humanitarian Crisis, same as the discussion above we think it is more suitable for MSF International, as they tend to focus more on the organisation as a whole, rather than the individual sectors.

The other KPIs listed in the report are very good, we really like the last KPI which is based on ethics, as in our report we did not make it an individual KPI, we combined it with independence. It is a really good idea to use the International Development Humanitarian Aid Code of Conduct as benchmark for MSF Australia and its staff.

We also found that you guys did not give some of the seven KPI’s names, this may be needed in order for easier identification.

Overall, your guys did a great job the report is very easy to follow.

Good work :)

Andrea & Katy

Dear Bronnie,

I can appreciate your point of view, and perhaps my comments as "the worst paper I've read" was a bit harsh. I can, however, assure you that my comment was not a personal attack on Zhe Khai Goh (New@y). I do not know this individual and my comments were in reflection to work that was submitted for review.

I took the time to read the complete report, and spent 2 hours formulating a calculated reply, with comments, criticism, suggestions for improvement and justification for my comments.

I'm sorry if I offended you but this is a tertiary competition conducted in the English language, and there were elements that I could not read or make sense of New@y's report.

Hey guys! Thanks for commenting on our report. Nice video! I liked your report’s format, but felt that it could have been enhanced by including the KPI formulas. I see that you have conducted good research on the external factors, yet felt that there were some misunderstandings regarding MSF Australia’s internal operations.

You have underlined in the introduction that you produced KPIs that specifically pertain to the objectives and operations of MSF Australia, yet I fail to see the connection between some of the KPIs and MSF Australia’s objectives and operations. As presented in MSF Australia’s Annual Report, the organisation is a non-operational partner of MSF International. The functions of MSF Australia include fundraising and recruitment. Our group confirmed the objectives of MSF Australia by calling the head office and discovered that MSF Australia does not have any involvement with the actual operational activities abroad. Therefore, the first two KPIs are irrelevant as they do not deal with the direct operations and objectives of MSF Australia. The KPIs are not useful to MSF Australia as they do not allow actionable changes to the MSF Australia’s processes that will enhance the organisation.

I agree with your assessment of the importance of HR KPIs, yet I did not quite understand the average rate of volunteer vacancy duration in the recruitment ratio. Could you elaborate on it more? The ratios seem to fulfil the basic HR needs of the organisation.

I agree with the experience and satisfaction KPI. I do not quite understand the future opportunities ratio, though. What opportunities are you referring to? What is the difference between the current satisfaction rating and the volunteers’ perception rating?

I do not fully agree with the awareness measures. Do you think it is a good idea to compare MSF Australia’s performance with that of other organisations? Since the objectives of each individual non-government organisations are different, I feel that the measure will not produce the required information that will aid the benchmarking of MSF Australia’s success. For instance, Oxfam Australia’s objectives are highly different than of MSF Australia in that Oxfam looks at the gender inequality of different nations and identifies them as a “crisis”. If this measure was applied to Oxfam, the outcome will reveal that Oxfam has a really bad awareness program where in reality it can be that the program is achieving its objectives.

I agree with the breakdown disclosure KPI; however you have not identified how the management can use the KPI to improve the performance. How will the management use the disclosed measures to aid MSF Australia’s performance? I also had difficulty understanding the “mandatory disclosure of conflict of interest by key staff and volunteers”. How can this be measured? I understand that the concept refers to agency theory. The problem with the agency theory is that it cannot be eliminated even given the current accounting procedures. How do you presume the conflict of interest can be eliminated?

Interested point on ethics, yet I feel that an ethics scorecard will not be directly connected to the operations of MSF Australia as they deal with the actual operations abroad. Of course, the impact of ethics will flow to MSF Australia’s efforts to raise donations and to recruit staff, yet the ethical issue is outside of MSF Australia’s specific scope of operations. MSF Australia will not be able to act on the measure as there is nothing that they can undertake with the scope of its operations.

Overall, good effort! You have definitely covered the main issues relating to MSF Australia. I liked the ideas covered in the report. I would appreciate if you respond to my questions so that I can fully comprehend your innovative thoughts. Thanks, guys.

Hi Sam,
In relation to Bronnie's comment, i do believe Bronnie has a point. In your discussion, you have made some valid points but your suggestions did not take or in any attempt, ask to clarify points made but rather made came to a direct yet harsh criticism. Unlike others in this competition who have questioned when points where unclear.
Your reply, saying it was a 'little harsh' to say it is the worse report you have ever seen is unjust, rude and inconsiderate. There are many students, internationally and domestically who have trouble with their english. You can't expect others to be to your standard when it is not their native tongue. That is, if the person is not from an english speaking background. The fact that it is a competition in the english language makes no difference, we all have the rights to enter regardless the level of english. Besides, your reoccurring comments on this individual’s english, seems you are the only one who has the problem with this aspect.
For KPI 1, misleading depends on how you interpret it. In regards to your suggestion of skewness when use statistical analysis, your inference is incorrect, there is more to statistical analysis then graphing technique. On the same token, ‘By the same logic, high donations do not necessarily correlate with high awareness. Apart from being skewed by single, passionate individuals donating larger sums, how does this measure account for public knowledge of MSF’s activities in the large proportion of people who do not donate,’ this is only to correct a certain extent, and you have also made a valid point, my report suggested that more donations necessarily means increased awareness, again you have to look beyond skewness in statistical analysis, your account for public knowledge hence goes beyond the measurement proposed by performance indicator. The measure of awareness for non-donators can realm into another KPI in itself I believe. For some reason, it seems like you like to ‘skew’ everything in relation to KPI 1.
In addition we can, look at various other statistical techniques such as hypothesis testing, chi squares, confidence, regression, in fact, depending on the data obtained and what we want to find out we can advance on an indepth analysis using various statistical procedures. Subsequently, your points take a narrow approach; hence your suggestions can only be narrowly viewed upon.
Looking at your lengthy and sub-critical replies to other teams, im sure you are set on winning this competition, you report is well formulated and structured and indeed it is very impressive as a contender. You’re a smart guy, as you have said, so im sure entering a competition for every one of your ideas to be heavily criticized, thrown back at you in saying how ‘’bad’ it is indeed, a good feeling? I’d say probably not, if i had the 2 hours to critique reports, guess who you would hear from. All the best mate, but don’t count on my vote to win this competition.
When you come to read my report, please give ‘concessions to my english’, for I am also from a non English speaking background.

Hi Sam,
In relation to Bronnie's comment, i do believe Bronnie has a point. In your discussion, you have made some valid points but your suggestions did not take or in any attempt, ask to clarify points made but rather made came to a direct yet harsh criticism. Unlike others in this competition who have questioned when points where unclear.
Your reply, saying it was a 'little harsh' to say it is the worse report you have ever seen is unjust, rude and inconsiderate. There are many students, internationally and domestically who have trouble with their english. You can't expect others to be to your standard when it is not their native tongue. That is, if the person is not from an english speaking background. The fact that it is a competition in the english language makes no difference, we all have the rights to enter regardless the level of english. Besides, your reoccurring comments on this individual’s english, seems you are the only one who has the problem with this aspect.
For KPI 1, misleading depends on how you interpret it. In regards to your suggestion of skewness when use statistical analysis, your inference is incorrect, there is more to statistical analysis then graphing technique. On the same token, ‘By the same logic, high donations do not necessarily correlate with high awareness. Apart from being skewed by single, passionate individuals donating larger sums, how does this measure account for public knowledge of MSF’s activities in the large proportion of people who do not donate,’ this is only to correct a certain extent, and you have also made a valid point, my report suggested that more donations necessarily means increased awareness, again you have to look beyond skewness in statistical analysis, your account for public knowledge hence goes beyond the measurement proposed by performance indicator. The measure of awareness for non-donators can realm into another KPI in itself I believe. For some reason, it seems like you like to ‘skew’ everything in relation to KPI 1.
In addition we can, look at various other statistical techniques such as hypothesis testing, chi squares, confidence, regression, in fact, depending on the data obtained and what we want to find out we can advance on an indepth analysis using various statistical procedures. Subsequently, your points take a narrow approach; hence your suggestions can only be narrowly viewed upon.
Looking at your lengthy and sub-critical replies to other teams, im sure you are set on winning this competition, you report is well formulated and structured and indeed it is very impressive as a contender. You’re a smart guy, as you have said, so im sure entering a competition for every one of your ideas to be heavily criticized, thrown back at you in saying how ‘’bad’ it is indeed, a good feeling? I’d say probably not, if i had the 2 hours to critique reports, guess who you would hear from. All the best mate, but don’t count on my vote to win this competition.
When you come to read my report, please give ‘concessions to my english’, for I am also from a non English speaking background.

Hi Pavel

Thanks for you comments.

I will specifically address independence and ethics here. My fellow group members will reply separately.

The key point that we are trying to make with our KPIs is that in order to improve relations with donors, an organisation must be transparent and accountable. The link between MSF Australia and international branches of MSF indicates that this cannot be done without consideration of the organisation as a whole. I also make the distinction between performance in terms of generating more donations and performance in terms of living up to the objectives and principles of the organisation.

The materials and information used in fundraising and recruitment by MSF Australia relate to the fieldwork undertaken by operational branches. Thus in order to maintain MSF's reputation as a credible humanitarian aid organisation and to attract new donors, MSF Australia must prove that it is in fact what it says it is…an independent medical humanitarian aid organisation. The breakdown of source of funding is a measure of independence and furthermore, disclosure provides donors and potential donors with evidence that MSF Australia (and internationally) is an independent organisation. Transparency in reporting sources of funding should help MSF to raise its profile, hence attracting the attention of potential donors who are bombarded with appeals from not-for-profit organisations.

The ACFID Code of Conduct contains a clause that involves the reporting of conflicts of interest. Conflicts of interest are important in any organisation, but are particularly important for organisations that pride themselves on their independence and ethics. While it may be difficult to measure conflict of interest per se, having a system in place which ensures all staff/volunteers are able to disclose any real or potential conflict of interest goes some way to being able to monitor and act on it.

In regards to ethics, it is my belief that ethics should apply to all organisations, particularly those involved in (or associated with in MSF Australia’s case) with humanitarian operations. MSF recruits volunteers from Australia and New Zealand and sends them to operations in various countries. Thus ensuring that its volunteers adhere to its charter and humanitarian ethical principles is critical. These ethical principles not only relate to fieldwork and volunteers but administrative staff also. The activities of MSF Australia and other branches that actively coordinate fieldwork are inextricably linked. Disclosure of organisations activities in line with their ethical principles should further attract the attention of potential donors. Accountability is particularly important for MSF Australia as it is raising funds for operations that it is not directly carrying out. Essentially MSF Australia cannot trade off the international MSF brand without ensuring a commitment to the organisation’s overarching principles.

Our KPIs for cost-effectiveness again relate to the idea that MSF Australia cannot be viewed without consideration of operational branches. For a not-for profit to be accountable to its donors, it should report where donated funds are going and what they are being used for. Highlighting the uses of funds to donors and giving them some idea of the cost-effectiveness of these operations is important for ensuring that donors are satisfied with their decision to contribute. Hence there is a benefit in reporting to donors that their donations contributed to an operation that was cost effective in helping those in need.

Renae

Hi Tung Tran,

Thanks for your comments.

At no time did I say it was the worse report I'd ever seen, I referred to it as the worse I had read in this competition. Regardless, I accept the fact the my words were too harsh and would rephrase them if I could edit the comment.

I do not, nor ever did, refute any individual's right to enter this competition based on English speaking ability. That is a misrepresentation of my comments.

My opinion is that the author should have had his work proof-read by a colleague with superior English skills, and I do not think that this point of view is unjust, rude or inconsiderate. I think in reading the comments a number of other respondents have found issue with clarity and comprehension.

Regarding my comments on KP 1:
The author mentioned the 'statistical analysis' would be carried out, but did not detail any measures of testing. You can rebut by example of 'skewness' and talk all day about different forms of statistical analysis, but that is also not the point. I raised it as an example for the author to detail just what statistical analysis HE was referring to. That is his job, not mine.

For the remainder of the report I have endeavored to make construct comments and offer suggestions (as I have done in many other reports). However, there were many instances where I could not understand the author because of grammar issues and thus could not comment.

My suggestion is still the same, the author should have had a third party proof read his work before submitting it.

Hi Pavel,

Thanks for your feedback. I will address the HR questions you raised...

The average days of volunteer vacancy duration investigates the number of days a volunteer role remains vacant until the appropriate personnel is recruited. For example, a doctor role and a nurse role may be vacant for 25 and 17 days respectively. Hence the average days of volunteer vacancy duration in this scenario would be: (25 + 17) / 2 = 21 days. MSF Australia would be striving to minimise this rating, and it therefore is a measure of performance and efficiency of MSF Australia recruitment strategy.

The future opportunities rating examine how volunteers perceive future opportunities within MSF. For example, a volunteer who commenced in a team member role may be able to advance to a project coordinator role, which offers more responsibility. We believe this is important as this incentive of development and greater responsibility would increase the positive experience and satisfaction of volunteers.

Current satisfaction rating vs. volunteers’ perception rating:

The current satisfaction rating specifically looks at whether an individual volunteer is currently satisfied with the project they undertake, specifically if they feel they have participated sufficiently in the specific project and whether or not they have been recognised accordingly by MSF. Therefore it evaluates the volunteers’ individual interaction with the current project.

Volunteers’ perception rating is similar to the current satisfaction rating, except that it focuses on potential redeployment of volunteers in the future. Hence, this measure looks at whether or not the individual would consider volunteering again, or recommend this volunteer role to others by determining how MSF can better improve on retaining the services of volunteers. Hence, this is a more forward looking measure of experiences and satisfaction of volunteers.

Hi Sam,

Valid point, i suggest, he should have gotten his proof read by a third person and likewise your comments proof read by a lay person, I would do the same.

Hi Andrea, Katy & Eleanor
Thanks for taking the time to read our report.

I’ll endeavour to answer each of your questions.

DALYs & Data requirements:
Thanks to work by the World Health Organisation, standardised data for DALY calculation has been produced for each country and by each disease. This can be found here:
http://www.who.int/healthinfo/bod/en/index.html
In referring to data difficulties, the issue in 3rd world countries can be the accuracy of this data, but I can only speculate on this. However, the fact it is compiled by such credible source is encouraging. That answers the question on where do you get the data for DALY calculations, and removes the cost factor that was raised as a potential barrier.
An excellent point was raised with reference to distinguishing KPIs for MSF Australia and MSF International, based on their operational differences. Allow me to refer you to a quote from the MSF Australia’s president, Rowan Gillies, in the 2006 Annual Report (pp 10):
“MSF Australia strengthened its ability to support operations over the year, with the development of the Project Unit as part of the operational directorate based in Paris, focusing on the quality of paediatric and obstetric care, as well as providing field support in those areas.

This was a functional result of the move over the last few years for MSF Australia to go beyond its current essential
tasks of providing high-quality field volunteers, significant financial support and communications back-up for operations.

This occurred in conjunction with a development in the relationship that MSF Australia has built with the various Operational Centres of MSF. We are now one of four sections (France, USA, Japan and Australia) who have responsibility for the operational direction based in Paris.”

http://www.msf.org.au/docs/reports/msfa_annual_report_06.pdf
This provided us with the evidence that MSF Australia’s efforts have, and are continuing to, extend to an operational level. With this in mind we have identified the all of our measures, especially the cost effectiveness KPIs, in the context this future trend, as per the innovation criteria.
Human Resources, Quality of Staff and Staff satisfaction:
As mentioned above, MSF Australia has both operational and non-operational areas and requires personnel to fill roles in each. Regardless of whether you are looking at operational or non-operational, MSF Australia or MSF International, we firmly believe that quality of staff and staff satisfaction are so fundamentally important that they apply all these organisations, in any context.
With regard to clarity in naming our KPIs, after re-reading our report I can see that you are correct, and we could have made them clearer.
Thanks so much for taking the time to read our work,

Kind regards

Sam