{"publication_status":"published","year":"2018","title":"Combining Crowd and Machines for Multi-predicate Item Screening","date_created":"2023-02-09T14:29:10Z","publication":"Proceedings of the ACM on Human-Computer Interaction","citation":{"short":"E. Krivosheev, F. Casati, M. Baez Gonzalez, B. Benatallah, Proceedings of the ACM on Human-Computer Interaction 2 (2018) 1–18.","alphadin":"<span style=\"font-variant:small-caps;\">Krivosheev, Evgeny</span> ; <span style=\"font-variant:small-caps;\">Casati, Fabio</span> ; <span style=\"font-variant:small-caps;\">Baez Gonzalez, Marcos</span> ; <span style=\"font-variant:small-caps;\">Benatallah, Boualem</span>: Combining Crowd and Machines for Multi-predicate Item Screening. In: <i>Proceedings of the ACM on Human-Computer Interaction</i> Bd. 2, Association for Computing Machinery (ACM) (2018), Nr. CSCW, S. 1–18","apa":"Krivosheev, E., Casati, F., Baez Gonzalez, M., &#38; Benatallah, B. (2018). Combining Crowd and Machines for Multi-predicate Item Screening. <i>Proceedings of the ACM on Human-Computer Interaction</i>, <i>2</i>(CSCW), 1–18. <a href=\"https://doi.org/10.1145/3274366\">https://doi.org/10.1145/3274366</a>","chicago":"Krivosheev, Evgeny, Fabio Casati, Marcos Baez Gonzalez, and Boualem Benatallah. “Combining Crowd and Machines for Multi-Predicate Item Screening.” <i>Proceedings of the ACM on Human-Computer Interaction</i> 2, no. CSCW (2018): 1–18. <a href=\"https://doi.org/10.1145/3274366\">https://doi.org/10.1145/3274366</a>.","ieee":"E. Krivosheev, F. Casati, M. Baez Gonzalez, and B. Benatallah, “Combining Crowd and Machines for Multi-predicate Item Screening,” <i>Proceedings of the ACM on Human-Computer Interaction</i>, vol. 2, no. CSCW, pp. 1–18, 2018.","bibtex":"@article{Krivosheev_Casati_Baez Gonzalez_Benatallah_2018, title={Combining Crowd and Machines for Multi-predicate Item Screening}, volume={2}, DOI={<a href=\"https://doi.org/10.1145/3274366\">10.1145/3274366</a>}, number={CSCW}, journal={Proceedings of the ACM on Human-Computer Interaction}, publisher={Association for Computing Machinery (ACM)}, author={Krivosheev, Evgeny and Casati, Fabio and Baez Gonzalez, Marcos and Benatallah, Boualem}, year={2018}, pages={1–18} }","ama":"Krivosheev E, Casati F, Baez Gonzalez M, Benatallah B. Combining Crowd and Machines for Multi-predicate Item Screening. <i>Proceedings of the ACM on Human-Computer Interaction</i>. 2018;2(CSCW):1-18. doi:<a href=\"https://doi.org/10.1145/3274366\">10.1145/3274366</a>","mla":"Krivosheev, Evgeny, et al. “Combining Crowd and Machines for Multi-Predicate Item Screening.” <i>Proceedings of the ACM on Human-Computer Interaction</i>, vol. 2, no. CSCW, Association for Computing Machinery (ACM), 2018, pp. 1–18, doi:<a href=\"https://doi.org/10.1145/3274366\">10.1145/3274366</a>."},"user_id":"245590","_id":"2369","status":"public","publisher":"Association for Computing Machinery (ACM)","author":[{"full_name":"Krivosheev, Evgeny","last_name":"Krivosheev","first_name":"Evgeny"},{"first_name":"Fabio","full_name":"Casati, Fabio","last_name":"Casati"},{"full_name":"Baez Gonzalez, Marcos","last_name":"Baez Gonzalez","orcid":"0000-0003-1666-2474","first_name":"Marcos","id":"250311"},{"first_name":"Boualem","last_name":"Benatallah","full_name":"Benatallah, Boualem"}],"issue":"CSCW","volume":2,"doi":"10.1145/3274366","type":"journal_article","publication_identifier":{"eissn":["2573-0142"]},"abstract":[{"text":"                  This paper discusses how crowd and machine classifiers can be efficiently combined to screen items that satisfy a set of predicates. We show that this is a recurring problem in many domains, present machine-human (hybrid) algorithms that screen items efficiently and estimate the gain over human-only or machine-only screening in terms of performance and cost. We further show how, given a new classification problem and a set of classifiers of unknown accuracy for the problem at hand, we can identify how to manage the cost-accuracy trade off by progressively determining if we should spend budget to obtain test data (to assess the accuracy of the given classifiers), or to train an ensemble of classifiers, or whether we should leverage the existing machine classifiers with the crowd, and in this case how to efficiently combine them based on their estimated characteristics to obtain the classification. We demonstrate that the techniques we propose obtain significant cost/accuracy improvements with respect to the leading classification algorithms.\r\n                ","lang":"eng"}],"date_updated":"2023-02-16T11:05:04Z","page":"1-18","intvolume":"         2","language":[{"iso":"eng"}]}