Jump to content

Panos Ipeirotis

Add links
From Wikipedia, the free encyclopedia
Panagiotis G. Ipeirotis
Παναγιώτης Ηπειρώτης
Born (1976-05-03) May 3, 1976 (age 50)[1]
Serres, Greece[2]
CitizenshipGreek, American
AwardsLagrange Prize (2015)[3]
ACM SIGKDD Test of Time Award (2020)[4]
Academic background
EducationUniversity of Patras (BSc)
Columbia University (MSc, PhD)
ThesisClassifying and Searching Hidden-Web Text Databases (2004)
Luis Gravano
Academic work
DisciplineComputer science
InstitutionsNew York University Stern School of Business
Main interests
Crowdsourcing; human computation; Data quality; Text mining
Websitewww.stern.nyu.edu/faculty/bio/panagiotis-ipeirotis

Panagiotis G. Ipeirotis (also known as Panos Ipeirotis; born May 3, 1976, in Serres, Greece) is a Greek-American computer scientist and the Merchants' Council Professor of Technology and Business at the New York University Stern School of Business.[5][6] His research focuses on data mining, crowdsourcing, human computation, and the economics of online information systems.[7]

Ipeirotis received the Lagrange Prize in Complex Systems (2015)[3] and the ACM SIGKDD Test of Time Award (2020).[4] His research on Amazon Mechanical Turk underpinned the Pew Research Center's 2016 methodology for estimating the platform's size and has informed reporting on crowdsourced labor in outlets including The New York Times and The Economist.[8][9][10]

In addition to his academic career, Ipeirotis co-founded the AI consulting firm Detectica in 2015, which was acquired by Compass, Inc. in 2019.[11] He has also held research positions at Google (2013–2014) and at Meta's Reality Labs division (2024–2025).[5]

Early life

[edit]

Ipeirotis was born on May 3, 1976, in Serres, Greece.[2] In 1994, he won a Gold Medal at the 8th National Greek Competition in Chemistry.[12] Later that year, he represented Greece at the 26th International Chemistry Olympiad in Oslo, Norway.[13]

Education

[edit]

Ipeirotis earned his Diploma in Computer Engineering and Informatics (CEID) from the University of Patras in 1999.[5] He pursued graduate studies at Columbia University under the supervision of Luis Gravano, receiving his M.Sc. in 2001, M.Phil. in 2003, and Ph.D. in Computer Science in 2004.[5]

Academic career

[edit]

Ipeirotis began his academic career as a graduate research assistant at Columbia University (1999–2004). In 2004, he joined the Department of Information, Operations, and Management Sciences at the New York University Stern School of Business as an Assistant Professor. He was promoted to Associate Professor in 2010 and Full Professor in 2016. He holds a courtesy appointment at the Courant Institute of Mathematical Sciences[14] and is an associated faculty member at the NYU Center for Data Science.[15]

Ipeirotis has served as Program Co-Chair for the 13th ACM Conference on Electronic Commerce in 2012,[16] as Technical Program Co-Chair for The Web Conference 2018 (WWW 2018),[17] and as an Area Chair for the AAAI Conference on Artificial Intelligence in 2021 and 2023.[18][19] He co-organized the Human Computation Workshop series from its founding edition in 2009 (co-located with SIGKDD) through 2012,[20][21][22][23] and served as General Co-Chair of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP) in 2015, after the series had grown into a stand-alone AAAI conference.[24] He has served on the editorial boards of Management Science, IEEE Transactions on Knowledge and Data Engineering, the INFORMS Journal on Computing, and the ACM Journal of Data and Information Quality,[25] and was a founding co-Editor-in-Chief of the journal Collective Intelligence, launched in 2022 as a collaboration between SAGE Publications, the Association for Computing Machinery, and Nesta.[26][27]

Ipeirotis has been the subject of media profiles in business and Greek-language outlets. Bloomberg Businessweek profiled him in 2011, describing his work on technology that combines human and machine intelligence,[28] and again in 2013, in a feature that called him the "data dude" of business analytics.[29] A 2014 Kathimerini interview discussed his career and conditions in Greek higher education.[30]

Academic lineage and doctoral students

[edit]

Ipeirotis's doctoral advisor Luis Gravano was himself a student of Hector Garcia-Molina at Stanford. According to the Mathematics Genealogy Project, Ipeirotis has advised seven doctoral students at NYU and Columbia.[31]

His doctoral students include Beibei Li (now at Carnegie Mellon University's Heinz College) and Marios Kokkodis, both of whom subsequently received INFORMS Information Systems Society early-career awards — the Sandra A. Slaughter Early Career Award in 2019[32] and the Gordon B. Davis Young Scholar Award in 2020.[33] Former postdoctoral researcher Djellel Difallah is Assistant Professor at New York University Abu Dhabi.[34]

Research

[edit]

Ipeirotis's research spans text mining, crowdsourcing, and the economics of online information systems. His work has been published in venues such as Management Science, Information Systems Research, and IEEE Transactions on Knowledge and Data Engineering.[5]

Text mining and web databases

[edit]

Ipeirotis's doctoral research at Columbia University, supervised by Luis Gravano, addressed the problem of accessing text databases on the deep web — content behind search interfaces that conventional web crawlers cannot index. An early 2001 paper introduced a query-probing method for categorizing hidden-web databases by topic without retrieving any documents,[35] which the thesis later generalized into the QProber system using machine learning classifiers to derive the probing queries.[36] In parallel, Ipeirotis contributed to research on data integration: a 2001 paper at the International Conference on Very Large Data Bases (VLDB), co-authored with Gravano, H. V. Jagadish, Nick Koudas, S. Muthukrishnan, and Divesh Srivastava, showed how approximate string joins — matching slightly different versions of the same entity across databases — could be implemented entirely within standard relational database systems, without specialized indexes.[37]

Building on this foundation, Ipeirotis published two award-winning papers on modeling and querying text databases. A 2005 study applied survival analysis techniques to model how the content of 152 web databases changed over a 52-week period, winning the Best Paper Award at the 21st IEEE International Conference on Data Engineering.[38] His 2006 paper "To Search or to Crawl?" presented a cost-based framework for choosing between querying and crawling text databases when executing information extraction tasks, and won the Best Paper Award at ACM SIGMOD.[39] In 2007, Ipeirotis received a National Science Foundation CAREER Award for the project "Towards a Text-Centric Database Management System," which funded research on query optimization for information extraction and on the economic analysis of textual content.[40]

Ipeirotis's subsequent research extended in two directions: crowdsourcing quality management, addressing the challenge of obtaining reliable labeled data from non-expert workers; and a collaboration with Anindya Ghose that used economic signals such as prices and sales as implicit indicators of textual content quality, an approach they termed "EconoMining."[7]

Crowdsourcing and human computation

[edit]

Ipeirotis's 2010 paper "Running Experiments on Amazon Mechanical Turk,"[41] co-authored with Gabriele Paolacci and Jesse Chandler, provided methodological guidance for using the platform — commonly abbreviated MTurk — in behavioral research.

A 2010 study by Ipeirotis estimated that approximately 40% of tasks posted by new requesters on MTurk involved spam or low-quality content, drawing media coverage.[42][43][44][45]

His 2010 demographic analysis of the MTurk workforce provided early data on worker motivations, geographic distribution, and income levels, and has been cited by researchers studying crowdsourcing platforms.[46][47] A follow-up study with former postdoctoral researcher Djellel Difallah and Elena Filatova, presented at the ACM International Conference on Web Search and Data Mining in 2018, tracked changes in the MTurk workforce over an eight-year period.[48] Media outlets have continued to draw on this body of work as concerns about the human labor underlying machine learning training data have grown, including coverage in Wired and The New York Times on automated responses on MTurk[49][50] and a 2026 Marketplace interview in which Ipeirotis discussed CAPTCHA completion and other routine online interactions as unrecognized AI-training-data labor.[51] The Financial Times cited his demographic and wage data on Mechanical Turk workers in a 2016 examination of the human workforce behind commercial artificial intelligence,[52] and The Wall Street Journal quoted him in 2017 on the continued reliance of large companies on human workers within their AI systems.[53] In 2020, Wired quoted him on an apparent influx of new workers onto Mechanical Turk during the COVID-19 pandemic, including workers based in Canada, Italy, and Brazil.[54]

This line of work also led to the development of quality management techniques for crowdsourcing. The 2008 paper "Get Another Label?," co-authored with Foster Provost and Victor S. Sheng, addressed the problem that training machine learning classifiers on labels from non-expert workers degrades model accuracy when those labels are noisy. The paper studied whether repeated labeling of uncertain items could improve classifier performance under noisy labels, and proposed selective repeated-labeling strategies for crowdsourced annotation. It received the Best Paper Runner-Up Award at the 2008 ACM SIGKDD Conference on Knowledge Discovery and Data Mining[55] and the SIGKDD Test of Time Award in 2020.[4] A 2025 survey of learning from crowdsourced noisy labels—the problem of combining unreliable labels from many non-expert workers—noted that large language models such as ChatGPT, Gemini, and Llama are fine-tuned on substantial volumes of crowdsourced labels, including through reinforcement learning from human feedback (RLHF) and direct preference optimization.[56]

ProPublica drew on Ipeirotis's research and blog posts in publishing its "Guide to Mechanical Turk" for investigative reporters, citing his work on data quality through multiple worker verification.[57] Ipeirotis has also commented on the use of crowdsourcing for content moderation; in a 2012 Foreign Policy article on image moderation, he described comparing the judgments of moderators from different regions as a way to reduce cultural bias.[58]

To support this research, Ipeirotis developed "MTurk Tracker," a set of tracking tools for the MTurk platform. The first tool tracked workforce demographics using capture-recapture techniques to estimate the size and composition of the workforce in near real-time. The second tool periodically downloaded all tasks posted on the platform, enabling analysis of the crowdsourcing marketplace's size and activity. The Pew Research Center's 2016 report "Research in the Crowdsourcing Age" drew on data from the task tracker, explicitly crediting "the online tool mturk-tracker, which is run by Dr. Panagiotis G. Ipeirotis."[8][59] The 2019 book Ghost Work, by Mary L. Gray and Siddharth Suri, drew on Ipeirotis's worker-demographics research to estimate the size of the platform's active workforce.[60]

During a 2013–2014 sabbatical at Google, Ipeirotis worked on crowdsourcing methods for improving the Google Knowledge Graph. He co-developed "Quizz," a gamified system that presented data collection as trivia quizzes targeted by domain expertise, using calibration questions with known answers to assess user reliability in real-time.[61][62] In a related paper with Chun How Tan, Eugene Agichtein, and Evgeniy Gabrilovich, Ipeirotis examined methods for predicting contribution quality in knowledge-base construction and curation.[63]

Ipeirotis also applied crowdsourcing methods to building training data for computer vision systems. With Pietro Perona and Serge Belongie — co-recipients with Ipeirotis of a 2012 Google Focused Research Award — he co-authored a 2015 paper at the IEEE Conference on Computer Vision and Pattern Recognition describing the construction of a large-scale dataset for fine-grained bird species recognition using citizen-scientist labels.[64]

In a 2015 article in the ACM Journal on Data and Information Quality, co-authored with Josh Attenberg and Foster Provost, Ipeirotis introduced "Beat the Machine," a framework in which crowdworkers were rewarded for finding examples that a trained classifier predicted incorrectly. The approach was proposed as a method for surfacing the "unknown unknowns" — systematic blind spots — of machine learning models.[65]

Data quality and record linkage

[edit]

Ipeirotis has published research on duplicate record detection and data quality. The 2007 survey "Duplicate Record Detection: A Survey," co-authored with Ahmed Elmagarmid and Vassilios Verykios, appeared in IEEE Transactions on Knowledge and Data Engineering and reviewed record linkage techniques across the database, statistics, and machine learning literatures.[66]

Online reputation and user-generated content

[edit]

As part of the EconoMining project, Ipeirotis and Anindya Ghose quantified the pricing power derived from textual content in product reviews; the work was discussed in the Financial Times.[67][68] A related 2011 study found that the quality of spelling and grammar in product reviews affects perceived helpfulness and product sales.[69] The finding was discussed in Forbes, Harvard Business Review, Slate, and the Freakonomics blog.[70][71][72][73]

[edit]

With Anindya Ghose and doctoral student Beibei Li, Ipeirotis developed text-mining and econometric methods for ranking products and hotels on online platforms. A 2011 paper introduced a theoretical model for product search that combined consumer review text with product characteristics to rank items by their predicted economic utility, receiving the Best Paper Award at the 20th International World Wide Web Conference.[74][75] A 2012 study in Marketing Science extended the framework to hotel ranking on travel search engines, combining mining of online reviews with crowdsourced ratings of hotel attributes.[76] A 2014 study in Management Science analyzed how the ordering of search results affects consumer click-through and platform revenue.[77]

Online labor markets

[edit]

With doctoral student Marios Kokkodis, Ipeirotis studied reputation systems and hiring dynamics in online labor markets such as oDesk (now Upwork). A 2015 study at the ACM International Conference on Web Search and Data Mining modeled employer hiring decisions using features such as worker reputation, prior experience, and bid characteristics.[78] Their work on reputation transferability modeled how a freelancer's performance ratings in one task category can predict outcomes in another, analyzing over one million transactions; it was published in Management Science in 2016.[79] A subsequent study applied reinforcement learning to recommend career paths to freelancers based on dynamic market demand, finding that the framework could increase both marketplace revenue and worker wages.[80] They also developed recommender systems for matching job applicants to employers on freelancing platforms.[81]

Applied research and industry

[edit]

Early industry work (2009–2014)

[edit]

Beginning in 2009, Ipeirotis served as an early advisor to Integral Ad Science (originally AdSafe Media), contributing to the development of machine learning systems for detecting inappropriate web content and advertising fraud.[82] In 2011, working with AdSafe engineers, he identified a click fraud scheme that used hidden iframes to generate fraudulent ad impressions, which he termed "traffic laundering."[83] The investigation, which MIT Technology Review headlined as "A Web Scam That Makes $500,000 a Month," was also reported by The Wall Street Journal and led to an FBI referral.[84][83]

In 2011, Ipeirotis co-founded Tagasauris, a media annotation startup where he served as Chief Scientist until 2013.[29] The company combined crowdsourcing with semantic graph technology to tag photo archives. The system identified approximately two dozen previously unidentified photographs from the 1973 filming of "American Graffiti" in the Magnum Photos archive, by having crowdworkers tag individuals and using machine learning to link them via shared film appearances.[85][86]

He also served as Academic-in-Residence at Upwork (then oDesk) in 2012 and as a Visiting Scientist at Google from 2013 to 2014.[5]

Policy research and World Bank

[edit]

In 2015, Ipeirotis collaborated with the World Bank on a report titled "The Global Opportunity in Online Outsourcing," which examined the potential of digital labor markets to provide employment pathways in developing nations.[87] Drawing on his empirical research from Mechanical Turk and oDesk, the report estimated the online outsourcing market would grow to $15–25 billion by 2020 and identified barriers preventing workers in the Global South from accessing these jobs, including lack of reliable internet infrastructure and the "reputation cold start" problem. The report has been cited in subsequent policy work by the RAND Corporation on online outsourcing as a youth-employment pathway in Indonesia.[88]

Detectica and Compass (2015–2022)

[edit]

In 2015, Ipeirotis co-founded Detectica with Foster Provost and Josh Attenberg, offering AI strategy consulting and machine learning solutions for business applications.[89] The company developed AI-driven compliance monitoring systems for financial institutions.[89]

Detectica was acquired by Compass, Inc. in November 2019.[11] At Compass, the Detectica team developed "Likely to Sell," a predictive analytics system that identifies properties likely to enter the market; Compass discussed the system as a revenue contributor on its 2022 and 2023 earnings calls.[90][91]

Meta Reality Labs (2024–2025)

[edit]

From 2024 to 2025, Ipeirotis was a visiting researcher at Meta's Reality Labs division, working on machine learning deployment for wearable devices.[5][92]

Public engagement and pedagogy

[edit]

Since 2008, Ipeirotis has maintained the blog "A Computer Scientist in a Business School,"[93] which has served as the original venue for several posts that subsequently drew independent media coverage, including his 2011 commentary on academic integrity and his 2025 description of an AI-driven oral examination format.

Academic integrity debates

[edit]

In July 2011, Ipeirotis published a post on his blog, titled "Why I will never pursue cheating again," describing his experience using Turnitin software to detect that about 20 percent of the students in his roughly 100-person "Information Technology in Business and Society" course had plagiarized.[93][94] The post criticized the administrative burden placed on faculty who enforce academic integrity policies and drew coverage in higher-education media, including Inside Higher Ed and The Chronicle of Higher Education.[94][95] Ipeirotis temporarily removed the post following concerns raised about whether quoted student communications could implicate the Family Educational Rights and Privacy Act (FERPA).[95]

Business Insider reported that Ipeirotis said his student evaluations declined after the plagiarism cases, contributing to a reduced annual raise;[96] NYU Stern vice dean Ingo Walter responded that faculty are not penalized for enforcing academic integrity and that student evaluations from such cases are excluded from review.[94] The incident prompted discussion about academic integrity enforcement, faculty incentives, and student privacy,[94][95] and was subsequently cited in Nature in a 2012 commentary on plagiarism detection in academia.[97]

Security vulnerability discovery

[edit]

In April 2012, Ipeirotis discovered a vulnerability in Google Spreadsheets that he termed a "Denial of Money" attack. While running a crowdsourcing experiment that required image classification, he uploaded 25,000 image URLs (hosted on his Amazon S3 bucket) into a Google Spreadsheet. Google's Feedfetcher bot repeatedly re-downloaded all images from his S3 bucket every time the spreadsheet was processed, generating a high volume of outgoing bandwidth. Ipeirotis reported receiving an Amazon Web Services bill of $1,177.76—approximately ten times his normal monthly rate.[98]

Coverage of the incident described it as illustrating a potential attack vector: a malicious actor could create a Google Sheet with thousands of links to a victim's large files, causing Google's servers to repeatedly request the files and resulting in either financial damage or a denial of service. The discovery was covered in technology publications including Wired and The Economist.[99][98][100][101] Amazon subsequently forgave the charges, and Google investigated the behavior.[98]

AI-powered assessment

[edit]

In December 2025, Ipeirotis developed an AI-driven oral examination system using ElevenLabs voice agents to address concerns about students submitting written assignments produced with generative AI tools. The system, which cost approximately $0.42 per student to administer, used a voice AI agent to call students and ask personalized questions about their submitted work.[102][103] An Associated Press feature in April 2026 reported on the approach as part of broader coverage of U.S. colleges adopting oral examinations in response to generative AI, quoting Ipeirotis on his rationale: "I don't trust written assignments anymore to be the result of actual thinking."[104] Ipeirotis and Konstantinos Rizakos described the system's design and evaluation in a 2026 preprint, in which the examinations used a multi-agent workflow and a panel of large language models to grade the resulting transcripts.[105] In a student survey reported by the NYU student newspaper Washington Square News, 70 percent of respondents agreed that the oral format better assessed their comprehension of course material, while 83 percent found it more stressful than written exams.[106] The approach was also discussed on the academic-philosophy site Daily Nous and the education-technology podcast EdTechnical.[107][108]

Awards and honors

[edit]
  • 2020: ACM SIGKDD Test of Time Award for Research (with Foster Provost and Victor S. Sheng). The Test of Time Award honors papers from approximately ten years prior that have had lasting impact on the field of knowledge discovery and data mining. The award recognized their 2008 paper "Get Another Label? Improving Data Quality and Data Mining Using Multiple, Noisy Labelers" "for their approach to selective acquisition of multiple labels."[4]
  • 2018: Named in a 2018 feature in the Greek newspaper Kathimerini as one of "45 under 45" Greek scientists — researchers born after 1974 whose publications, per the newspaper's accompanying methodology, ranked among the most-cited in their fields.[109][110]
  • 2015: Lagrange Prize in Complex Systems (shared with Jure Leskovec), awarded by the CRT Foundation and ISI Foundation. The prize citation recognized Ipeirotis's "pioneering work in the field of crowdsourcing and human computation" and research "[combining] economic, social psychology and automatic text analysis methods to quantify the economic value of the content generated by users on the Internet."[3][111]
  • 2014: Best Paper Award, Second AAAI Conference on Human Computation and Crowdsourcing (HCOMP), for "STEP: A Scalable Testing and Evaluation Platform".[112][113]
  • 2014: INFORMS Information Systems Society (ISS) Best Paper Award (Management Science), for "Deriving the Pricing Power of Product Features by Mining Consumer Reviews" (2011).[114]
  • 2012: Google Focused Research Award ($1.5 million), with Serge Belongie and Pietro Perona.[115]
  • 2011: Best Paper Award, 20th International World Wide Web Conference (WWW 2011), for "Towards a Theory Model for Product Search".[74]
  • 2008: Best Paper Runner-Up Award, 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2008), for "Get Another Label? Improving Data Quality and Data Mining Using Multiple, Noisy Labelers" (with Foster Provost and Victor S. Sheng).[55]
  • 2007: National Science Foundation CAREER Award, "Towards a Text-Centric Database Management System."[40]
  • 2006: Best Paper Award, ACM International Conference on Management of Data (SIGMOD 2006) — "To Search or to Crawl? Towards a Query Optimizer for Text-Centric Tasks".[39]
  • 2005: Best Paper Award, 21st IEEE International Conference on Data Engineering (ICDE 2005), for "Modeling and Managing Content Changes in Text Databases".[38]

Selected publications

[edit]
  • Sheng, Victor S.; Provost, Foster; Ipeirotis, Panagiotis G. (2008). Get another label? Improving data quality and data mining using multiple, noisy labelers. Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. doi:10.1145/1401890.1401965. (KDD Test of Time Award, 2020)
  • Paolacci, Gabriele; Chandler, Jesse; Ipeirotis, Panagiotis G. (2010). "Running experiments on Amazon Mechanical Turk". Judgment and Decision Making. 5 (5): 411–419. doi:10.1017/S1930297500002205. hdl:1765/31983.
  • Ghose, Anindya; Ipeirotis, Panagiotis G. (2011). "Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics". IEEE Transactions on Knowledge and Data Engineering. 23 (10): 1498–1512. doi:10.1109/TKDE.2010.188.
  • Archak, Nikolay; Ghose, Anindya; Ipeirotis, Panagiotis G. (2011). "Deriving the pricing power of product features by mining consumer reviews". Management Science. 57 (8): 1485–1509. doi:10.1287/mnsc.1110.1370.
  • Elmagarmid, Ahmed K.; Ipeirotis, Panagiotis G.; Verykios, Vassilios S. (2007). "Duplicate record detection: A survey". IEEE Transactions on Knowledge and Data Engineering. 19 (1): 1–16. doi:10.1109/TKDE.2007.250581.

References

[edit]
  1. ↑ "Panagiotis G. Ipeirotis". ORCID. Retrieved 2026-02-12.
  2. 1 2 "Πάνος Ηπειρώτης: Ο Σερραίος καθηγητής που διαπρέπει στη Νέα Υόρκη" [Panos Ipeirotis: The Serres-born professor excelling in New York]. Aveserres.gr (in Greek). 2025-04-11. Retrieved 2026-02-04.
  3. 1 2 3 "Press Release: Lagrange Prize 2015". ISI Foundation. 2015-10-06. Archived from the original on 2015-10-06. Retrieved 2025-12-22.
  4. 1 2 3 4 "2020 SIGKDD Test of Time Award Winners". ACM SIGKDD. Retrieved 2025-12-22.
  5. 1 2 3 4 5 6 7 "NYU Stern - Panagiotis Ipeirotis - Merchants' Council Professor of Technology, Operations, and Statistics". www.stern.nyu.edu. Retrieved 2025-12-22.
  6. ↑ "TOPS | Full-time Faculty - NYU Stern". www.stern.nyu.edu. Retrieved 2025-12-22.
  7. 1 2 Ghose, Anindya; Ipeirotis, Panagiotis (2009). "The EconoMining project at NYU: Studying the economic value of user-generated content on the internet". Journal of Revenue and Pricing Management. 8 (2–3): 241–246. doi:10.1057/rpm.2008.56. S2CID 154923072.
  8. 1 2 "Methodology: Mechanical Turk". Pew Research Center. 2016-07-11. Retrieved 2026-02-04.
  9. ↑ Segal, David (2011-05-22). "A Rave, a Pan, or Just a Fake?". The New York Times. Retrieved 2026-02-04.
  10. ↑ "Turks of the World, Unite!". The Economist. 2011-05-24. Retrieved 2026-02-04.
  11. 1 2 "Real-Estate Brokerage Compass Buys AI Startup Detectica". The Wall Street Journal. 2019-11-19. Retrieved 2025-12-22.
  12. ↑ "8η Ολυμπιάδα Χημείας" [8th Chemistry Olympiad]. Χημικά Χρονικά (Chemical Chronicles) (in Greek). 56 (8): 233. 1994. Archived from the original (PDF) on 2026-02-13. Retrieved 2026-02-02.
  13. ↑ "26th IChO 1994 Results". International Chemistry Olympiad. Archived from the original on 2026-01-10. Retrieved 2026-02-02.
  14. ↑ "Faculty". NYU Courant Institute of Mathematical Sciences. Retrieved 2025-12-22.
  15. ↑ "Associated Faculty". NYU Center for Data Science. Retrieved 2025-12-22.
  16. ↑ "13th ACM Conference on Electronic Commerce". 13th ACM Conference on Electronic Commerce. Retrieved 2026-02-05.
  17. ↑ "Research Tracks CFP – TheWebConf 2018". The Web Conference 2018. Retrieved 2026-01-31.
  18. ↑ "AAAI-21 Area Chairs". Association for the Advancement of Artificial Intelligence. Retrieved 2026-05-13.
  19. ↑ "AAAI-23 Area Chairs". Association for the Advancement of Artificial Intelligence. Retrieved 2026-05-13.
  20. ↑ "Organizing Committee – First Human Computation Workshop (HComp 2009)". Human Computation Workshop. Retrieved 2026-05-20.
  21. ↑ "Organizers – Second Human Computation Workshop (HComp 2010)". AAAI Workshop on Human Computation. Retrieved 2026-05-20.
  22. ↑ "Organizers – Third Human Computation Workshop (HCOMP 2011)". AAAI Workshop on Human Computation. Retrieved 2026-05-20.
  23. ↑ "Organizers – Fourth Human Computation Workshop (HCOMP 2012)". AAAI Workshop on Human Computation. Retrieved 2026-05-20.
  24. ↑ "HCOMP 2015: Organizers". AAAI Conference on Human Computation and Crowdsourcing. Retrieved 2026-01-31.
  25. ↑ "Long Biographical Note". Panos Ipeirotis (personal website). Retrieved 2026-05-20.
  26. ↑ "Editorial Board". Collective Intelligence. SAGE Publications. Retrieved 2026-01-31.
  27. ↑ "SAGE, Association for Computing Machinery Announce Collective Intelligence, New Open Access Journal in Collaboration with Nesta". Association for Computing Machinery. 2020-08-18. Retrieved 2026-01-31.
  28. ↑ Weise, Karen (2011-11-10). "Humans Plus Computers Equals Better Crowdsourcing". Bloomberg Businessweek. Archived from the original on 2014-11-09. Retrieved 2026-01-27.
  29. 1 2 "The Data Dude: NYU Stern's Panos Ipeirotis". Bloomberg Businessweek. 2013-01-24. Retrieved 2026-02-04.
  30. ↑ "Παναγιώτης Ηπειρώτης: «Θαυμάζω όσους παράγουν έργο στα πανεπιστήμια με τις συνθήκες που επικρατούν»" [Panagiotis Ipeirotis: "I admire those who produce work in universities under the prevailing conditions"]. Kathimerini (in Greek). 2014-07-21. Retrieved 2026-02-03.
  31. ↑ "Panagiotis Ipeirotis - Mathematics Genealogy Project". Mathematics Genealogy Project. Retrieved 2026-02-03.
  32. ↑ "ISS Sandra A. Slaughter Early Career Award". INFORMS. Retrieved 2026-02-03.
  33. ↑ "ISS Gordon B. Davis Young Scholar Award". INFORMS. Retrieved 2026-02-03.
  34. ↑ "Djellel Difallah". NYU Abu Dhabi. Retrieved 2026-02-03.
  35. ↑ Ipeirotis, Panagiotis G.; Gravano, Luis; Sahami, Mehran (2001). Probe, count, and classify: categorizing hidden web databases. Proceedings of the 2001 ACM SIGMOD International Conference on Management of Data. ACM. pp. 67–78. doi:10.1145/375663.375671.
  36. ↑ Gravano, Luis; Ipeirotis, Panagiotis G.; Sahami, Mehran (2003). "QProber: A System for Automatic Classification of Hidden-Web Databases". ACM Transactions on Information Systems. 21 (1): 1–41. doi:10.1145/635484.635485.
  37. ↑ Gravano, Luis; Ipeirotis, Panagiotis G.; Jagadish, H. V.; Koudas, Nick; Muthukrishnan, S.; Srivastava, Divesh (2001). Approximate string joins in a database (almost) for free. Proceedings of the 27th International Conference on Very Large Data Bases. Morgan Kaufmann. pp. 491–500.
  38. 1 2 "ICDE 2005 Best Paper Awards". 21st IEEE International Conference on Data Engineering. Archived from the original on 2005-04-10. Retrieved 2025-12-22.{{cite web}}: CS1 maint: bot: original URL status unknown (link)
  39. 1 2 "SIGMOD Best Paper Award". ACM SIGMOD. Retrieved 2025-12-22.
  40. 1 2 "Award Search: Award #0643846 – CAREER: Towards a Text-Centric Database Management System". National Science Foundation. Retrieved 2025-12-22.
  41. ↑ Paolacci, Gabriele; Chandler, Jesse; Ipeirotis, Panagiotis G. (2010). "Running experiments on Amazon Mechanical Turk". Judgment and Decision Making. 5 (5): 411–419. doi:10.1017/S1930297500002205. hdl:1765/31983. ISSN 1930-2975.
  42. ↑ Walsh, Bryan (2011-01-31). "On Crowdsourcing: Pennies for Your Thoughts". TIME. Archived from the original on 2018-08-10. Retrieved 2026-02-03.
  43. ↑ Mims, Christopher. "How Mechanical Turk is Broken". MIT Technology Review. Archived from the original on 2011-12-12. Retrieved 2025-12-22.
  44. ↑ "40% Of Amazon's Mechanical Turk Is Spam". Business Insider. 2010-12-17. Retrieved 2025-12-22.
  45. ↑ Rosenwald, Michael S. (2011-01-30). "How you and Google are losing the battle against spam in search results". The Washington Post. ISSN 0190-8286. Retrieved 2025-12-22.
  46. ↑ Ipeirotis, Panagiotis G. (2010). "Demographics of Mechanical Turk". NYU Stern School of Business. Retrieved 2026-02-04.
  47. ↑ Fort, Karën; Adda, Gilles; Cohen, K. Bretonnel (2011). "Amazon Mechanical Turk: Gold Mine or Coal Mine?". Computational Linguistics. 37 (2). MIT Press: 413–420. doi:10.1162/COLI_a_00057.
  48. ↑ Difallah, Djellel; Filatova, Elena; Ipeirotis, Panos (2018). Demographics and Dynamics of Mechanical Turk Workers. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. ACM. pp. 135–143. doi:10.1145/3159652.3159661.
  49. ↑ Dreyfuss, Emily (2018-08-17). "A Bot Panic Hits Amazon Mechanical Turk". Wired. Retrieved 2026-02-03.
  50. ↑ Newman, Andy (2019-11-15). "I Found Work on an Amazon Website. I Made 97 Cents an Hour". The New York Times. Retrieved 2026-02-04.
  51. ↑ Ben-Achour, Sabri (2026-05-11). "You may be helping train AI and not even know it". Marketplace. Retrieved 2026-05-19.
  52. ↑ Hook, Leslie (2016-10-26). "The humans behind Mechanical Turk's artificial intelligence". Financial Times. Retrieved 2026-06-25.
  53. ↑ Mims, Christopher (2017-11-12). "Without Humans, Artificial Intelligence Is Still Pretty Stupid". The Wall Street Journal. Retrieved 2026-06-25.
  54. ↑ Simonite, Tom (2020-04-23). "Newly Unemployed, and Labeling Photos for Pennies". Wired. Retrieved 2026-06-25.
  55. 1 2 "2008 SIGKDD Best Paper Award". ACM SIGKDD. Retrieved 2026-05-13.
  56. ↑ Ibrahim, Shahana; Traganitis, Panagiotis A.; Fu, Xiao; Giannakis, Georgios B. (2025). "Learning from Crowdsourced Noisy Labels: A Signal Processing Perspective". IEEE Signal Processing Magazine. 42 (3): 84–106. arXiv:2407.06902. doi:10.1109/MSP.2025.3572636.
  57. ↑ "ProPublica's Guide to Mechanical Turk". ProPublica. 2010. Retrieved 2026-02-03.
  58. ↑ Anderson, Sulome (2012-11-14). "Why did Facebook censor this photograph?". Foreign Policy. Retrieved 2026-06-25.
  59. ↑ "The size of the Mechanical Turk marketplace". Pew Research Center. 2016-07-11. Retrieved 2026-02-04.
  60. ↑ Gray, Mary L.; Suri, Siddharth (2019). Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Houghton Mifflin Harcourt. ISBN 978-1328566249.
  61. ↑ Ipeirotis, Panagiotis G.; Gabrilovich, Evgeniy (2014). Quizz: targeted crowdsourcing with a billion (potential) users. Proceedings of the 23rd International Conference on World Wide Web. ACM. pp. 143–154. arXiv:1506.01062. doi:10.1145/2566486.2567988.
  62. ↑ Slawski, Bill (2014-08-25). "Good Bye Knowledge Graph, Hello Google Knowledge Vault?". Go Fish Digital Blog. Retrieved 2026-02-04.
  63. ↑ Tan, Chun How; Agichtein, Eugene; Ipeirotis, Panos; Gabrilovich, Evgeniy (2014). Trust, but verify: predicting contribution quality for knowledge base construction and curation. Proceedings of the Seventh ACM International Conference on Web Search and Data Mining. ACM. pp. 553–562. doi:10.1145/2736277.2741681.
  64. ↑ Van Horn, Grant; Branson, Steve; Farrell, Ryan; Haber, Scott; Barry, Jessie; Ipeirotis, Panos; Perona, Pietro; Belongie, Serge (2015). Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE. pp. 595–603. doi:10.1109/CVPR.2015.7298658.
  65. ↑ Attenberg, Josh; Ipeirotis, Panagiotis G.; Provost, Foster (2015). "Beat the Machine: Challenging Humans to Find a Predictive Model's "Unknown Unknowns"". ACM Journal on Data and Information Quality. 6 (1): 1–17. doi:10.1145/2700832.
  66. ↑ Elmagarmid, Ahmed K.; Ipeirotis, Panagiotis G.; Verykios, Vassilios S. (2007). "Duplicate Record Detection: A Survey". IEEE Transactions on Knowledge and Data Engineering. 19 (1): 1–16. doi:10.1109/TKDE.2007.250581. ISSN 1558-2191.
  67. ↑ Archak, Nikolay; Ghose, Anindya; Ipeirotis, Panagiotis G. (2011). "Deriving the Pricing Power of Product Features by Mining Consumer Reviews". Management Science. 57 (8): 1485–1509. doi:10.1287/mnsc.1110.1370. ISSN 0025-1909.
  68. ↑ "Something for the Weekend". Financial Times. 2011-10-28. Retrieved 2026-02-04.
  69. ↑ Ghose, Anindya; Ipeirotis, Panagiotis G. (2011). "Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteristics". IEEE Transactions on Knowledge and Data Engineering. 23 (10): 1498–1512. doi:10.1109/TKDE.2010.188. ISSN 1558-2191.
  70. ↑ Savitz, Eric (2011-06-08). "Zapos, Zappoz, or Zappos: Why Typos Are Good For Your Brand". Forbes. Archived from the original on 2011-07-11. Retrieved 2025-12-22.
  71. ↑ "The Value of Teaching Your Customers How to Spell". Harvard Business Review. 2011-05-26. Archived from the original on 2013-07-18. Retrieved 2025-12-22.
  72. ↑ Agger, Michael (2011-05-10). "Awsum Shoes!". Slate. ISSN 1091-2339. Retrieved 2025-12-22.
  73. ↑ "Does Reviewer Quality Matter?". Freakonomics. 2011-04-12. Retrieved 2026-06-25.
  74. 1 2 "The Twentieth International WWW Conference (WWW 2011) – Best Paper Award". International World Wide Web Conference Archives (IW3C2). Retrieved 2025-12-22.
  75. ↑ Li, Beibei; Ghose, Anindya; Ipeirotis, Panagiotis G. (2011). Towards a theory model for product search. Proceedings of the 20th International Conference on World Wide Web. ACM. pp. 327–336. doi:10.1145/1963405.1963453.
  76. ↑ Ghose, Anindya; Ipeirotis, Panagiotis G.; Li, Beibei (2012). "Designing Ranking Systems for Hotels on Travel Search Engines by Mining User-Generated and Crowdsourced Content". Marketing Science. 31 (3): 493–520. doi:10.1287/mksc.1110.0700.
  77. ↑ Ghose, Anindya; Ipeirotis, Panagiotis G.; Li, Beibei (2014). "Examining the Impact of Ranking on Consumer Behavior and Search Engine Revenue". Management Science. 60 (7): 1632–1654. doi:10.1287/mnsc.2013.1828.
  78. ↑ Kokkodis, Marios; Papadimitriou, Panagiotis; Ipeirotis, Panagiotis G. (2015). Hiring Behavior Models for Online Labor Markets. Proceedings of the Eighth ACM International Conference on Web Search and Data Mining. ACM. pp. 223–232. doi:10.1145/2684822.2685299.
  79. ↑ Kokkodis, Marios; Ipeirotis, Panagiotis G. (2016). "Reputation Transferability in Online Labor Markets". Management Science. 62 (6): 1687–1706. doi:10.1287/mnsc.2015.2217.
  80. ↑ Kokkodis, Marios; Ipeirotis, Panagiotis G. (2021). "Demand-Aware Career Path Recommendations: A Reinforcement Learning Approach". Management Science. 67 (7): 4362–4383. doi:10.1287/mnsc.2020.3727.
  81. ↑ Kokkodis, Marios; Ipeirotis, Panagiotis G. (2023). "The Good, the Bad, and the Unhirable: Recommending Job Applicants in Online Labor Markets". Management Science. 69 (11): 6969–6987. doi:10.1287/mnsc.2023.4690.
  82. ↑ "About Us". Detectica. Retrieved 2026-01-31. Panos served as an early advisor of Integral Ad Science.
  83. 1 2 Mims, Christopher (2011-03-21). "A Web Scam That Makes $500,000 a Month". MIT Technology Review. Retrieved 2026-01-31.
  84. ↑ "Off Screen, Porn Sites Trick Advertisers". The Wall Street Journal. 2011-03-18. Retrieved 2026-01-31.
  85. ↑ "Hidden Treasure: Lost Photos From the Set of American Graffiti". Wired. 2011-07-01. Retrieved 2026-02-03.
  86. ↑ Mims, Christopher (2011-06-20). "How Six Degrees of Kevin Bacon Unearthed Lost Photos From American Graffiti". MIT Technology Review. Retrieved 2026-02-03.
  87. ↑ "The Global Opportunity in Online Outsourcing". World Bank Open Knowledge Repository. 2015. Retrieved 2026-02-03.
  88. ↑ "Online Outsourcing: Prospects for Increasing Youth Employment and Reducing Poverty in Indonesia". RAND Corporation. Retrieved 2026-02-03.
  89. 1 2 "Detectica - Enabling organizations to adopt effective AI strategies". Techweek. 2019-12-19. Retrieved 2025-12-22.
  90. ↑ "Compass, Inc. (COMP) Q4 2021 Earnings Call Transcript". The Motley Fool. 2022-02-17. Retrieved 2025-12-22. In 2021, $151 million gross commission revenue was from listings that the likely to sell tool recommended to our agents before the listing was created. We expect this number to exceed $400 million in 2022.
  91. ↑ "Compass, Inc. (COMP) Q1 2023 Earnings Call Transcript". The Motley Fool. 2023-05-09. Retrieved 2025-12-22. Since launching in 2020, we have handed out millions of recommendations that be attributed more than $100 million of annual incremental revenue to these recommendations over each of the last two years.
  92. ↑ "CCI Seminar Series 2024-25: Panos Ipeirotis". MIT Center for Collective Intelligence. Retrieved 2026-02-02.
  93. 1 2 "A Computer Scientist in a Business School". Behind the Enemy Lines. Retrieved 2025-12-22.
  94. 1 2 3 4 Jaschik, Scott (2011-07-22). "Who Is Punished for Plagiarism?". Inside Higher Ed. Retrieved 2026-01-31.
  95. 1 2 3 Parry, Marc (2011-07-21). "NYU Prof Vows Never to Probe Cheating Again—and Faces a Backlash". The Chronicle of Higher Education. Retrieved 2026-01-31.
  96. ↑ "NYU Professor Catches 20% of His Students Cheating, And He's The One Who Pays For It". Business Insider. 2011-07-20. Retrieved 2026-02-03.
  97. ↑ "Science publishing: How to stop plagiarism". Nature. 481 (7378): 21–23. 2012-01-04. doi:10.1038/481021a. Retrieved 2026-02-04.
  98. 1 2 3 McMillan, Robert (2012-04-27). "Oops! Amazon Web Services Customer Unleashes 'Denial of Money' Attack – on Himself". Wired. Retrieved 2026-02-04.
  99. ↑ "Twice shy, once bitten". The Economist. 2012-05-28. Retrieved 2026-02-04.
  100. ↑ Bort, Julie (2012-04-30). "An Amazon Customer Accidentally Ran Up A $1,200 Bill In Just 10 Minutes". Business Insider. Retrieved 2026-02-04.
  101. ↑ "Accidentally invented – Dos attack using Google Spreadsheets". The Hacker News. 2012-04-28. Retrieved 2026-02-04.
  102. ↑ ""An NYU professor who hates that students' work reads like McKinsey memos held AI oral exams to 'fight fire with fire'"". Business Insider. 2026-01-05. Retrieved 2026-01-31.
  103. ↑ Bastian, Matthias (2026-01-06). "NYU professor fights AI cheating with AI-powered oral exams that cost 42 cents per student". The Decoder. Retrieved 2026-01-31.
  104. ↑ Gecker, Jocelyn (2026-04-22). "Perfect homework, blank stares: Why colleges are turning to oral exams to combat AI". Associated Press. Retrieved 2026-05-19.
  105. ↑ Ipeirotis, Panagiotis G.; Rizakos, Konstantinos (2026-03-18). "Scalable and Personalized Oral Assessments Using Voice AI". arXiv:2603.18221 [cs.CY].
  106. ↑ Kemiktarak, Selin (2026-01-28). "Stern professors use AI-led oral exams to test student comprehension". Washington Square News. Retrieved 2026-06-25.
  107. ↑ Weinberg, Justin (2026-01-12). "Fighting AI with AI". Daily Nous. Retrieved 2026-06-25.
  108. ↑ Hills, Libby; Henkel, Owen (2026-03-12). "AI broke take-home assignments. Can it fix them too?". EdTechnical. Retrieved 2026-06-25.
  109. ↑ "Έλληνες στην κορυφή της επιστήμης" [Greeks at the Top of Science]. Kathimerini (in Greek). 2018. Retrieved 2026-02-03.
  110. ↑ "Professor Panos Ipeirotis is Recognized as One of '45 Under 45' Greek Scientists Producing Highest-Impact Publications". NYU Stern. Retrieved 2026-02-02.
  111. ↑ "Research Highlight: Panos Ipeirotis - 2015 CRT Foundation Lagrange Prize". NYU Stern. Retrieved 2026-01-31.
  112. ↑ "Past Meetings". AAAI Conference on Human Computation and Crowdsourcing. Retrieved 2025-12-22.
  113. ↑ Christoforaki, Maria; Ipeirotis, Panagiotis (2014-11-05). STEP: A Scalable Testing and Evaluation Platform. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing. Vol. 2. AAAI.
  114. ↑ "Management Science—2014 Best Paper Awards". Management Science. INFORMS. 2015-01-15. doi:10.1287/mnsc.2014.2122. Retrieved 2025-12-22.
  115. ↑ "$1.5 Million Google Research Grant Awarded to Professor Panos Ipeirotis & Colleagues". NYU Stern. 2013-02-22. Retrieved 2025-12-22.
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Panos_Ipeirotis&oldid=1378092440"