SlideShare una empresa de Scribd logo
1 de 29
Descargar para leer sin conexión
Data Science of the
KDD ‘14 Review Process
Jure Leskovec (Stanford) and
Wei Wang (UCLA)
Joint work with
Jason Hirshman and David Zeng (Stanford)
KDD 2014 Research Track
Statistics
KDD 2014 Program
Largest KDD program ever:
• 151 research papers (20% growth over KDD’13)
• 43 industry & govt. papers (30% growth)
• 26 workshops (75% growth)
• 11 tutorials (83% growth)
Program highlights:
• Paper spotlights early morning (8:15am)
• Oral presentations (Mon-Wed)
• Posters at the reception (Tue night)
KDD 2014 Research Track
• 1036 submissions from 2600 authors
– 42% increase over KDD ’13
• 151 papers:
– Acceptance rate
14.6%
0
200
400
600
800
1000
1200
2000 2005 2010 2015
KDD year
Numberofsubmissions
KDD Reviewing Process
46 Senior PC members + 340 PC members
• 2971 reviews in total
(Rough) Acceptance rule:
• Raw review score AND Standardized review score AND Raw
meta-review AND Standardized meta-review score ≥ Weak
Accept
• 110 papers matched (immediate accepts)
• Remaining papers were discussed with meta-reviewers and
final decisions were made
Submissions per Country
Acceptance Rate per Country
Acceptance by Subject Area
Predicting Paper Acceptance
Features Used Accuracy
Random Guessing 0.50
Paper Abstract 0.57
Author Status (Past paper counts) 0.64
Author Status (DBLP graph connectivity) 0.61
Author Status (Counts + Graph) 0.65
Reviewer (Similarity, Graph distance to authors) 0.60
All (Abstract, Author Status, and Reviewer) 0.65
Predicting Paper Acceptance
from the Review Text
Features Used
Paper:
Accepted?
Review:
Score > 0?
Random Guessing 0.50 0.50
Review Text 0.68 0.72
Review Text + Numeric Score
(Novelty, Presentation)
0.77 0.77
Human Reading of Review Text 0.88 0.73
I’m submitting a paper:
What correlates with acceptance?
Academia + Industry Papers do Better
Submissions per Author: 5 is best!
No benefit in submitting >5 papers!
Having more authors (seems to) help
It is the most experienced author
that matters!
What insights can we gain on
the review process?
Most reviews are Weak Rejects
More granularity is needed at the
Weak Reject / Weak Accept level
Reviewagreeswiththefinaloutcome
Review length is a good determinant
of a review’s influence/quality
Reviewagreeswiththefinaloutcome
Shorter reviews are used for
clear accepts and rejects
Never review co-author’s papers
The Curse of the Review
Submission Deadline
Over 50% reviews submitted in the last 5 days
Over 20% reviews submitted in the last 24 hours
10% of reviews
submitted late
Ratings increase near the deadline
Weak Rejects
increase while
Rejects decrease
Reviews submitted late are less likely
to agree with final outcome
Late reviews are shorter
Review quality drops: Accuracy of
predicting score from review text
Conclusions
• To get your papers accepted to KDD:
– Collaborate in multidisciplinary teams
– Have a senior author on board
– Do not submit more than 5 papers
• To improve KDD community standards:
– Avoid Weak Reject/Weak Accept scores
– Write longer and clearer reviews
– Submit reviews early!

Más contenido relacionado

Destacado

Big Data per Madee 7 at Digital Accademia
Big Data per Madee 7 at Digital AccademiaBig Data per Madee 7 at Digital Accademia
Big Data per Madee 7 at Digital AccademiaGianluigi Cogo
 
KDD HARVEST - COMPANY PRESENTATION-1-2
KDD HARVEST - COMPANY PRESENTATION-1-2KDD HARVEST - COMPANY PRESENTATION-1-2
KDD HARVEST - COMPANY PRESENTATION-1-2Kapil Sharma
 
KDD Analytics 2014 - Experts in Marketing Analytics
KDD Analytics 2014 - Experts in Marketing AnalyticsKDD Analytics 2014 - Experts in Marketing Analytics
KDD Analytics 2014 - Experts in Marketing AnalyticsBoulder Equity Analytics
 
marketing plan of nestle cerelac
marketing plan of nestle cerelacmarketing plan of nestle cerelac
marketing plan of nestle cerelacabdullah khan
 
How-To Align Marketing & Sales to Boost Revenue (Infographic)
How-To Align Marketing & Sales to Boost Revenue (Infographic)How-To Align Marketing & Sales to Boost Revenue (Infographic)
How-To Align Marketing & Sales to Boost Revenue (Infographic)Brian Downard
 
Advertising Plan of Nestle Milk Pack (Relaunch)
Advertising Plan of Nestle Milk Pack (Relaunch)Advertising Plan of Nestle Milk Pack (Relaunch)
Advertising Plan of Nestle Milk Pack (Relaunch)Syed Ahmed Owais
 
Marketing plan of nestle
Marketing plan of nestleMarketing plan of nestle
Marketing plan of nestleabdullah khan
 
How to Make Money Selling Physical Products - Steve Chou
How to Make Money Selling Physical Products - Steve ChouHow to Make Money Selling Physical Products - Steve Chou
How to Make Money Selling Physical Products - Steve ChouLeslie Samuel
 

Destacado (11)

Big Data per Madee 7 at Digital Accademia
Big Data per Madee 7 at Digital AccademiaBig Data per Madee 7 at Digital Accademia
Big Data per Madee 7 at Digital Accademia
 
KDD HARVEST - COMPANY PRESENTATION-1-2
KDD HARVEST - COMPANY PRESENTATION-1-2KDD HARVEST - COMPANY PRESENTATION-1-2
KDD HARVEST - COMPANY PRESENTATION-1-2
 
KDD Analytics 2014 - Experts in Marketing Analytics
KDD Analytics 2014 - Experts in Marketing AnalyticsKDD Analytics 2014 - Experts in Marketing Analytics
KDD Analytics 2014 - Experts in Marketing Analytics
 
Random walk on Graphs
Random walk on GraphsRandom walk on Graphs
Random walk on Graphs
 
Health Benefits of Pomegranate
Health Benefits of PomegranateHealth Benefits of Pomegranate
Health Benefits of Pomegranate
 
Aseptic packaging
Aseptic packagingAseptic packaging
Aseptic packaging
 
marketing plan of nestle cerelac
marketing plan of nestle cerelacmarketing plan of nestle cerelac
marketing plan of nestle cerelac
 
How-To Align Marketing & Sales to Boost Revenue (Infographic)
How-To Align Marketing & Sales to Boost Revenue (Infographic)How-To Align Marketing & Sales to Boost Revenue (Infographic)
How-To Align Marketing & Sales to Boost Revenue (Infographic)
 
Advertising Plan of Nestle Milk Pack (Relaunch)
Advertising Plan of Nestle Milk Pack (Relaunch)Advertising Plan of Nestle Milk Pack (Relaunch)
Advertising Plan of Nestle Milk Pack (Relaunch)
 
Marketing plan of nestle
Marketing plan of nestleMarketing plan of nestle
Marketing plan of nestle
 
How to Make Money Selling Physical Products - Steve Chou
How to Make Money Selling Physical Products - Steve ChouHow to Make Money Selling Physical Products - Steve Chou
How to Make Money Selling Physical Products - Steve Chou
 

Similar a Data Science view of the KDD 2014

CJUS 750Data Analysis Grading RubricCriteriaLevels of Ac
CJUS 750Data Analysis Grading RubricCriteriaLevels of AcCJUS 750Data Analysis Grading RubricCriteriaLevels of Ac
CJUS 750Data Analysis Grading RubricCriteriaLevels of AcVinaOconner450
 
Do Citations and Readership Predict Excellent Publications?
Do Citations and Readership Predict Excellent Publications?Do Citations and Readership Predict Excellent Publications?
Do Citations and Readership Predict Excellent Publications?Dasha Herrmannova
 
UX London Collaborative Research Workshop
UX London Collaborative Research WorkshopUX London Collaborative Research Workshop
UX London Collaborative Research WorkshopErika Hall
 
Presenting your Research at the ECTEL Doctoral Consortium
Presenting your Research at the ECTEL Doctoral ConsortiumPresenting your Research at the ECTEL Doctoral Consortium
Presenting your Research at the ECTEL Doctoral ConsortiumChristian Glahn
 
Frontiers' Collaborative Review
Frontiers' Collaborative ReviewFrontiers' Collaborative Review
Frontiers' Collaborative ReviewFrontiersIn
 
Publishing Trends In Materials Science
Publishing Trends In Materials SciencePublishing Trends In Materials Science
Publishing Trends In Materials Sciencematerialsmx
 
Enhancing Engagement with Subject Specific Revision Sessions
Enhancing Engagement with Subject Specific Revision SessionsEnhancing Engagement with Subject Specific Revision Sessions
Enhancing Engagement with Subject Specific Revision SessionsSHU Learning & Teaching
 
DSD-INT 2021 The choice - A workshop for modelers
DSD-INT 2021 The choice - A workshop for modelersDSD-INT 2021 The choice - A workshop for modelers
DSD-INT 2021 The choice - A workshop for modelersDeltares
 
Teac powerpoint 1
Teac powerpoint 1Teac powerpoint 1
Teac powerpoint 1hblewis
 
UXPA 2023: UX research: Optimizing collaboration with project research sponsors
UXPA 2023: UX research: Optimizing collaboration with project research sponsorsUXPA 2023: UX research: Optimizing collaboration with project research sponsors
UXPA 2023: UX research: Optimizing collaboration with project research sponsorsUXPA International
 
Trend Spotting Workshop
Trend Spotting WorkshopTrend Spotting Workshop
Trend Spotting WorkshopMarieke Guy
 
How to write a 4* impact case study
How to write a 4* impact case studyHow to write a 4* impact case study
How to write a 4* impact case studyMark Reed
 
School of Nursing FNP MSN5300 Advanced Nursing Inquiry and.docx
School of Nursing FNP   MSN5300 Advanced Nursing Inquiry and.docxSchool of Nursing FNP   MSN5300 Advanced Nursing Inquiry and.docx
School of Nursing FNP MSN5300 Advanced Nursing Inquiry and.docxlillie234567
 
Visualizing Student Feedback
Visualizing Student FeedbackVisualizing Student Feedback
Visualizing Student FeedbackMargus Niitsoo
 

Similar a Data Science view of the KDD 2014 (20)

CJUS 750Data Analysis Grading RubricCriteriaLevels of Ac
CJUS 750Data Analysis Grading RubricCriteriaLevels of AcCJUS 750Data Analysis Grading RubricCriteriaLevels of Ac
CJUS 750Data Analysis Grading RubricCriteriaLevels of Ac
 
Do Citations and Readership Predict Excellent Publications?
Do Citations and Readership Predict Excellent Publications?Do Citations and Readership Predict Excellent Publications?
Do Citations and Readership Predict Excellent Publications?
 
UX London Collaborative Research Workshop
UX London Collaborative Research WorkshopUX London Collaborative Research Workshop
UX London Collaborative Research Workshop
 
Behind the scenes of peer review
Behind the scenes of peer reviewBehind the scenes of peer review
Behind the scenes of peer review
 
Presenting your Research at the ECTEL Doctoral Consortium
Presenting your Research at the ECTEL Doctoral ConsortiumPresenting your Research at the ECTEL Doctoral Consortium
Presenting your Research at the ECTEL Doctoral Consortium
 
Frontiers' Collaborative Review
Frontiers' Collaborative ReviewFrontiers' Collaborative Review
Frontiers' Collaborative Review
 
Publishing Trends In Materials Science
Publishing Trends In Materials SciencePublishing Trends In Materials Science
Publishing Trends In Materials Science
 
Enhancing Engagement with Subject Specific Revision Sessions
Enhancing Engagement with Subject Specific Revision SessionsEnhancing Engagement with Subject Specific Revision Sessions
Enhancing Engagement with Subject Specific Revision Sessions
 
DSD-INT 2021 The choice - A workshop for modelers
DSD-INT 2021 The choice - A workshop for modelersDSD-INT 2021 The choice - A workshop for modelers
DSD-INT 2021 The choice - A workshop for modelers
 
Teac powerpoint 1
Teac powerpoint 1Teac powerpoint 1
Teac powerpoint 1
 
UXPA 2023: UX research: Optimizing collaboration with project research sponsors
UXPA 2023: UX research: Optimizing collaboration with project research sponsorsUXPA 2023: UX research: Optimizing collaboration with project research sponsors
UXPA 2023: UX research: Optimizing collaboration with project research sponsors
 
agile vs. traditional methodologies
agile vs. traditional methodologies agile vs. traditional methodologies
agile vs. traditional methodologies
 
Trend Spotting Workshop
Trend Spotting WorkshopTrend Spotting Workshop
Trend Spotting Workshop
 
Interview workshop slides final
Interview workshop slides  finalInterview workshop slides  final
Interview workshop slides final
 
How to write a 4* impact case study
How to write a 4* impact case studyHow to write a 4* impact case study
How to write a 4* impact case study
 
School of Nursing FNP MSN5300 Advanced Nursing Inquiry and.docx
School of Nursing FNP   MSN5300 Advanced Nursing Inquiry and.docxSchool of Nursing FNP   MSN5300 Advanced Nursing Inquiry and.docx
School of Nursing FNP MSN5300 Advanced Nursing Inquiry and.docx
 
Rapid Review Methods in a COVID-19 World
Rapid Review Methods in a COVID-19 WorldRapid Review Methods in a COVID-19 World
Rapid Review Methods in a COVID-19 World
 
ACL 2017 Opening Session
ACL 2017 Opening Session ACL 2017 Opening Session
ACL 2017 Opening Session
 
Visualizing Student Feedback
Visualizing Student FeedbackVisualizing Student Feedback
Visualizing Student Feedback
 
HCI_Lecture04.pptx
HCI_Lecture04.pptxHCI_Lecture04.pptx
HCI_Lecture04.pptx
 

Último

The market for cross-border mortgages in Europe
The market for cross-border mortgages in EuropeThe market for cross-border mortgages in Europe
The market for cross-border mortgages in Europe321k
 
Understanding the Impact of video length on student performance
Understanding the Impact of video length on student performanceUnderstanding the Impact of video length on student performance
Understanding the Impact of video length on student performancePrithaVashisht1
 
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j
 
Paul Martin (Gartner) - Show Me the AI Money.pdf
Paul Martin (Gartner) - Show Me the AI Money.pdfPaul Martin (Gartner) - Show Me the AI Money.pdf
Paul Martin (Gartner) - Show Me the AI Money.pdfdcphostmaster
 
Enabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsEnabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsNeo4j
 
Stochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxStochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxjkmrshll88
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptxFurkanTasci3
 
Data Collection from Social Media Platforms
Data Collection from Social Media PlatformsData Collection from Social Media Platforms
Data Collection from Social Media PlatformsMahmoud Yasser
 
Unleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMUnleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMMarco Wobben
 
2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-Profits2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-ProfitsTimothy Spann
 
Deloitte+RedCross_Talk to your data with Knowledge-enriched Generative AI.ppt...
Deloitte+RedCross_Talk to your data with Knowledge-enriched Generative AI.ppt...Deloitte+RedCross_Talk to your data with Knowledge-enriched Generative AI.ppt...
Deloitte+RedCross_Talk to your data with Knowledge-enriched Generative AI.ppt...Neo4j
 
How to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentHow to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentAggregage
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptxFurkanTasci3
 
PPT for Presiding Officer.pptxvvdffdfgggg
PPT for Presiding Officer.pptxvvdffdfggggPPT for Presiding Officer.pptxvvdffdfgggg
PPT for Presiding Officer.pptxvvdffdfggggbhadratanusenapati1
 
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j
 
Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1bengalurutug
 
Microeconomic Group Presentation Apple.pdf
Microeconomic Group Presentation Apple.pdfMicroeconomic Group Presentation Apple.pdf
Microeconomic Group Presentation Apple.pdfmxlos0
 
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...ferisulianta.com
 
Using DAX & Time-based Analysis in Data Warehouse
Using DAX & Time-based Analysis in Data WarehouseUsing DAX & Time-based Analysis in Data Warehouse
Using DAX & Time-based Analysis in Data WarehouseThinkInnovation
 

Último (20)

Target_Company_Data_breach_2013_110million
Target_Company_Data_breach_2013_110millionTarget_Company_Data_breach_2013_110million
Target_Company_Data_breach_2013_110million
 
The market for cross-border mortgages in Europe
The market for cross-border mortgages in EuropeThe market for cross-border mortgages in Europe
The market for cross-border mortgages in Europe
 
Understanding the Impact of video length on student performance
Understanding the Impact of video length on student performanceUnderstanding the Impact of video length on student performance
Understanding the Impact of video length on student performance
 
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
 
Paul Martin (Gartner) - Show Me the AI Money.pdf
Paul Martin (Gartner) - Show Me the AI Money.pdfPaul Martin (Gartner) - Show Me the AI Money.pdf
Paul Martin (Gartner) - Show Me the AI Money.pdf
 
Enabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsEnabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge Graphs
 
Stochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxStochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptx
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
 
Data Collection from Social Media Platforms
Data Collection from Social Media PlatformsData Collection from Social Media Platforms
Data Collection from Social Media Platforms
 
Unleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMUnleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IM
 
2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-Profits2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-Profits
 
Deloitte+RedCross_Talk to your data with Knowledge-enriched Generative AI.ppt...
Deloitte+RedCross_Talk to your data with Knowledge-enriched Generative AI.ppt...Deloitte+RedCross_Talk to your data with Knowledge-enriched Generative AI.ppt...
Deloitte+RedCross_Talk to your data with Knowledge-enriched Generative AI.ppt...
 
How to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentHow to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product Development
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
 
PPT for Presiding Officer.pptxvvdffdfgggg
PPT for Presiding Officer.pptxvvdffdfggggPPT for Presiding Officer.pptxvvdffdfgggg
PPT for Presiding Officer.pptxvvdffdfgggg
 
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
 
Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1
 
Microeconomic Group Presentation Apple.pdf
Microeconomic Group Presentation Apple.pdfMicroeconomic Group Presentation Apple.pdf
Microeconomic Group Presentation Apple.pdf
 
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
 
Using DAX & Time-based Analysis in Data Warehouse
Using DAX & Time-based Analysis in Data WarehouseUsing DAX & Time-based Analysis in Data Warehouse
Using DAX & Time-based Analysis in Data Warehouse
 

Data Science view of the KDD 2014

  • 1. Data Science of the KDD ‘14 Review Process Jure Leskovec (Stanford) and Wei Wang (UCLA) Joint work with Jason Hirshman and David Zeng (Stanford)
  • 2. KDD 2014 Research Track Statistics
  • 3. KDD 2014 Program Largest KDD program ever: • 151 research papers (20% growth over KDD’13) • 43 industry & govt. papers (30% growth) • 26 workshops (75% growth) • 11 tutorials (83% growth) Program highlights: • Paper spotlights early morning (8:15am) • Oral presentations (Mon-Wed) • Posters at the reception (Tue night)
  • 4. KDD 2014 Research Track • 1036 submissions from 2600 authors – 42% increase over KDD ’13 • 151 papers: – Acceptance rate 14.6% 0 200 400 600 800 1000 1200 2000 2005 2010 2015 KDD year Numberofsubmissions
  • 5. KDD Reviewing Process 46 Senior PC members + 340 PC members • 2971 reviews in total (Rough) Acceptance rule: • Raw review score AND Standardized review score AND Raw meta-review AND Standardized meta-review score ≥ Weak Accept • 110 papers matched (immediate accepts) • Remaining papers were discussed with meta-reviewers and final decisions were made
  • 9. Predicting Paper Acceptance Features Used Accuracy Random Guessing 0.50 Paper Abstract 0.57 Author Status (Past paper counts) 0.64 Author Status (DBLP graph connectivity) 0.61 Author Status (Counts + Graph) 0.65 Reviewer (Similarity, Graph distance to authors) 0.60 All (Abstract, Author Status, and Reviewer) 0.65
  • 10. Predicting Paper Acceptance from the Review Text Features Used Paper: Accepted? Review: Score > 0? Random Guessing 0.50 0.50 Review Text 0.68 0.72 Review Text + Numeric Score (Novelty, Presentation) 0.77 0.77 Human Reading of Review Text 0.88 0.73
  • 11. I’m submitting a paper: What correlates with acceptance?
  • 12. Academia + Industry Papers do Better
  • 14. No benefit in submitting >5 papers!
  • 15. Having more authors (seems to) help
  • 16. It is the most experienced author that matters!
  • 17. What insights can we gain on the review process?
  • 18. Most reviews are Weak Rejects
  • 19. More granularity is needed at the Weak Reject / Weak Accept level Reviewagreeswiththefinaloutcome
  • 20. Review length is a good determinant of a review’s influence/quality Reviewagreeswiththefinaloutcome
  • 21. Shorter reviews are used for clear accepts and rejects
  • 23. The Curse of the Review Submission Deadline
  • 24. Over 50% reviews submitted in the last 5 days Over 20% reviews submitted in the last 24 hours 10% of reviews submitted late
  • 25. Ratings increase near the deadline Weak Rejects increase while Rejects decrease
  • 26. Reviews submitted late are less likely to agree with final outcome
  • 27. Late reviews are shorter
  • 28. Review quality drops: Accuracy of predicting score from review text
  • 29. Conclusions • To get your papers accepted to KDD: – Collaborate in multidisciplinary teams – Have a senior author on board – Do not submit more than 5 papers • To improve KDD community standards: – Avoid Weak Reject/Weak Accept scores – Write longer and clearer reviews – Submit reviews early!

Notas del editor

  1. Country of the paper is given by the mode author nationality. Only countries with more than 10 submissions are shown (except South Korea, which had 0 acceptance)
  2. Subject areas were based on a field that authors tagged their papers with. Only subject areas with more than 50 submissions are shown.
  3. On balanced dataset (we subsampled negative examples)
  4. On balanced dataset (we subsampled negative examples)
  5. Academic: All authors of paper are affiliated with a university Industry: All authors of paper are affiliated with industry Mixed: Paper has authors affiliated with both universities and industry.