<?xml version="1.0" encoding="utf-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.0 20120330//EN" "JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="article">
<front>
    <journal-meta>
        <journal-id journal-id-type="publisher-id">INFEDU</journal-id>
        <journal-title-group>
            <journal-title>Informatics in Education</journal-title>
        </journal-title-group>
        <issn pub-type="epub">1648-5831</issn>
        <issn pub-type="ppub">1648-5831</issn>
        <publisher>
            <publisher-name>VU</publisher-name>
        </publisher>
    </journal-meta>
    <article-meta>
                <article-id pub-id-type="publisher-id">INFE072</article-id>
                        <article-id pub-id-type="doi">10.15388/infedu.2006.01</article-id>
                        <article-categories>
            <subj-group subj-group-type="heading">
                <subject>Article</subject>
            </subj-group>
        </article-categories>
                        <title-group>
            <article-title>Random Factors in IOI 2005 Test Case Scoring</article-title>
        </title-group>
                        <contrib-group>
                                        <contrib contrib-type="author">
                                                <name>
                    <surname>CORMACK</surname>
                    <given-names>Gordon</given-names>
                </name>
                                <email xlink:href="mailto:gvcormac@uwaterloo.ca">gvcormac@uwaterloo.ca</email>
                                                <xref ref-type="aff" rid="j_INFEDU_aff_000"/>
                                            </contrib>
                        <aff id="j_INFEDU_aff_000">David R. Cheriton School of Computer Science, University of Waterloo Waterloo Ontario N2L 3G1, Canada</aff>
                                </contrib-group>
                                                                            <volume>5</volume>
                                <issue>1</issue>
                                    <fpage>5</fpage>
                        <lpage>14</lpage>
						<pub-date pub-type="epub">
                        <day>15</day>
                                    <month>04</month>
                        <year>2006</year>
        </pub-date>
                                                        <abstract>
                        <p>We examine the precision with which the cumulative score from a suite of test cases ranks participants in the International Olympiad in Informatics (IOI). Our concern is the ability of these scores to reflect achievement at all levels, as opposed to reflecting chance or arbitrary factors involved in composing the test suite. Test cases are assumed to be drawn from an infinite population of similar cases; variance in standardized rank is estimated by the bootstrap method and used to compute confidence intervals which contain the hypothetical true ranking with 95% probability. We examine the relative contribution of easy (so-called fifty-percent rule) cases and hard cases to the overall ranking. Empirical results based on IOI 2005 suggest that easy and hard cases are both material to the ranking, but the proportion of each is unimportant.</p>
                    </abstract>
                <kwd-group>
            <label>Keywords</label>
                        <kwd>informatics olympiad</kwd>
                        <kwd>programming contest</kwd>
                        <kwd>classical test theory</kwd>
                    </kwd-group>
    </article-meta>
</front>
</article>
