2019 USENIX Annual Technical Conference will take place July 10–12, 2019, at the Hyatt Regency Lake Washington in Renton, WA, USA. Authors of papers published in PACMPL Issue OOPSLA 2020 will present their work in the OOPSLA track of the SPLASH virtual conference in November. Chengyu Wang. This site uses Akismet to reduce spam. Artifact reviewers can then center their reviews / evaluation around these specific claims, though the reviewers will still consider whether the provided evidence is adequate to support claims that the artifact works. Common issues in the kick-the-tires phase included: As with last year, the single most effective way to avoid these sorts of issues ahead of time is to run the instructions independently on a fresh machine, VM, or Docker container. PACMPL Issue OOPSLA 2019 seeks contributions on all aspects of programming languages and software engineering. Gabriele Prato, Ella Charlaix and. Publication date: November 2020. Missing dependencies, or poor documentation of dependencies. Alan Snyder's appendix to the OOPSLA'91 proceedings on "How to Get Your Paper Accepted at OOPSLA" is very accurate. OOPSLA 2020 : Conference on Object-Oriented Programming Systems, Languages,and Applications Conference Series : Conference on Object-Oriented Programming Systems, Languages, and Applications Link: https://2020 If the alternative tool crashes on a subset of the inputs, simply note this expected behavior. A Reusable badge is given when reviewers feel the artifact is particularly well packaged, documented, designed, etc. The past several years shows this is feasible, but has costs. However, this biases reviews in favor of those with the funding for that (which can easily run bills up to several thousand USD). Post was not sent - check your email addresses! One hundred nine papers were approved out of 302 submissions, amounting to a 36% acceptance rate. Papers may target any stage of software development, including requirements, modeling, prototyping, design, implementation, generation, analysis, … This leads to a scramble every year to recognize which artifacts have these significant requirements, and to try to rebalance them to reviewers with existing access to possibly-suitable systems. Assistant Professor of Computer Science at City University of New York (CUNY) Hunter College, Home » Research » Papers » Actor concurrency study paper accepted at OOPSLA 2020. It is a forum for presenting original results in foundational aspects of … This has the additional benefit of still rewarding artifacts which perhaps were “close” to achieving a Functional designation. Vol. The papers below have been accepted for publication at OSDI '20. At the 23.8% acceptance rate of CHI 2019 (and imagining mean scores were the only criterion), all papers with a score greater than 3.0 would be accepted (647 papers). It may be possible to apply through various providers’ research credits programs, though it might also be useful to include this as part of sponsorship requests for future editions of OOPSLA. nominations for the Artifact Evaluation Committee (AEC). For more information on artifact reviewing, consult the 2020 calls for artifacts: https://2020.splashcon.org/track/splash-2020-Artifacts#Call-for-Artifacts. The overall number of submissions, however, increased substantially, from 44 last year (a 50% increase), which led to a last-minute scramble to grow the reviewer pool from 30 to 50 PhD students and post-docs, who wrote 200 reviews. Write this for readers who have a deep interest in your work and are studying it to improve it or compare against it. In such cases, all available benchmarks should be included. Please contact Colin Gordon and Anders Møller if you have any questions. PACMPL (OOPSLA) seeks contributions on all aspects of programming languages and software engineering. For an artifact to be accepted, it must support all the main claims made in the paper. There is value in leaving the reusability criteria open-ended, as reusability often means something very different for machine-checked proofs vs. proof-of-concept compilers vs. dynamic analysis tools. If the artifact claims to outperform a related system in some way (in time, accuracy, etc.) Please see the results in the Chairs’ Report. After decisions on the Functional and Reusable badges have been made, the AEC Chairs can award an additional badge to those accepted artifacts that make their artifact durably available: Available: This badge may only be awarded to artifacts judged functional. We believe it is worth decoupling the Phase 2 deadline from artifact evaluation to permit more time for artifact reviewing. Papers that go through the Artifact Evaluation process successfully will receive a seal of approval printed on the first page of the paper. Here are some links to conference content and a report on the conference. OOPSLA 2010 Research Papers October 17 to 20 Reno/Tahoe Nevada, USA www.splashcon.org Paper Submission Deadline: … This is recommended. Authors of papers published in PACMPL Issue OOPSLA 2019 will present their work at OOPSLA in Athens. Examples included generating a list of warnings without documenting which were true vs. false positives, and generating large tables of numbers that were presented graphically in the paper without providing a way to generate analogous visualizations. This submission is voluntary. Artifacts involving an AEC chair must be unambiguously accepted (they may not be borderline), and they may not be considered for the distinguished artifact award. Reviewers will follow all the steps in the guide during an initial kick-the-tires phase. Papers Awards Workshops Town Hall Socials Login Show/Hide Subject Areas Browse Papers Paper Visualization Showing papers for . If. One hundred nine papers were approved out of 302 submissions, amounting to a 36% acceptance rate. An artifact can be awarded a functional badge if the artifact supports all claims made in the paper, possibly excluding some minor claims if there are very good reasons they cannot be supported. A non-exclusive list of justifiable deviations includes: In some cases, the artifact may require specialized hardware (e.g., a CPU with a particular new feature, or a specific class of GPU, or a cluster of GPUs). Oct 19, 2020 COLING’2020 attracted an unprecedented number of submissions, in fact more than … OSDI '20 Accepted Papers We at USENIX assert that Black lives matter: Read the USENIX Statement on Racism and Black, African-American, and African Diaspora Inclusion . Papers may target any stage of software development, including requirements, modeling, prototyping, design, implementation, generation, analysis, verification, testing, evaluation, maintenance, and reuse … Prior experience with artifact evaluation (as a submitter or reviewer) is a plus, but also not required. Authors of papers with accepted artifacts are encouraged to make these materials publicly available upon publication of the proceedings, by including them as “source materials” in the ACM Digital Library. Onward! Actor concurrency study paper accepted at OOPSLA 2020, Click to email this to a friend (Opens in new window), Click to share on Twitter (Opens in new window), Click to share on Facebook (Opens in new window), Click to share on LinkedIn (Opens in new window), Click to share on Reddit (Opens in new window), Accepted EAPLS best paper award at virtual ETAPS 2020, Yiming passes the second exam – Raffi Khatchadourian, Allan successfully defends his thesis – Raffi Khatchadourian, Received EAPLS best paper award at FASE 2020 – Raffi Khatchadourian, Annie accepted to Yale University – Raffi Khatchadourian, Attribution-NonCommercial-ShareAlike 4.0 International. When packaging your artifact, please keep in mind: a) how accessible you are making your artifact to other researchers, and b) the fact that the AEC members will have a limited time in which to make an assessment of each artifact. . Please use a widely available compressed archive format such as ZIP (.zip), tar and gzip (.tgz), or tar and bzip2 (.tbz2). Papers may target any stage of software development, including requirements, modeling, prototyping, design, implementation, generation, analysis, verification, testing, evaluation, maintenance, and reuse of software systems. Tags: actors, concurrency, empirical software engineering, empirical study, new paper, oopsla. Welcome to SPLASH 2020, the ACM SIGPLAN conference on Systems, Programming, Languages, and Applications: Software for Humanity. In practice, it results in only 6 weeks for artifact reviewing, end to end. Powered by CUNY. If you are looking for a well-documented object-oriented framework to try your method, check-out this JUnit 3.8 documentation. and the other system was used to generate new numbers for the paper (e.g., an existing tool was run on new benchmarks not considered by the corresponding publication), artifacts should include a version of that related system, and instructions for reproducing the numbers used for comparison as well. This means that authors will not know who reviewed their papers, and reviewers will not know who authored the papers they review. A Verified Space Cost Semantics for CakeML Programs, Effects as Capabilities: Effect Handlers and Lightweight Effect Polymorphism, Eliminating Abstraction Overhead of Java Stream Pipelines using Ahead-of-Time Program Optimization, Finding Bugs in Database Systems via Query Partitioning, Formulog: Datalog for SMT-Based Static Analysis, Guided Linking: Dynamic Linking Without the Costs, Hidden Inheritance: An Inline Caching Design for TypeScript Performance, Igloo: Soundly Linking Compositional Refinement and Separation Logic for Distributed System Verification, Inter-Theory Dependency Analysis for SMT String Solvers, Interactive Synthesis of Temporal Specifications from Examples and Natural Language, Just-in-Time Learning for Inductive Program Synthesis, Learning Graph-based Heuristics for Pointer Analysis without Handcrafting Application-Specific Features, Learning-based Controlled Concurrency Testing, Multiparty Motion Coordination: From Choreographies to Robotics Programs, Perfectly Parallel Fairness Certification of Neural Networks, Precise Inference of Expressive Units of Measurement Types, Precise Static Modeling of Ethereum ``Memory'', Programming and Reasoning with Partial Observability, Projection-based Runtime Assertions for Testing and Debugging Quantum Programs, Regex Matching with Counting-Set Automata, Resolution as Intersection Subtyping via Modus Ponens, Scaling Exact Inference for Discrete Probabilistic Programs, Statically Verified Refinements for Multiparty Protocols, StreamQL: A Query Language for Processing Streaming Time Series, Testing Differential Privacy with Dual Interpreters, The Anchor Verifier for Blocking and Non-Blocking Concurrent Software, Verifying Replicated Data Types with Typeclass Refinements in Liquid Haskell, ιDOT: A DOT Calculus with Object Initialization, https://2020.splashcon.org/track/splash-2020-Artifacts#Call-for-Artifacts, Learning-Based Controlled Concurrency Testing, Recommendations for Future Artifact Evaluations, August 8: Authors of papers accepted in Phase 1 submit artifacts, August 15-18: Authors may respond to issues found following kick-the-tires instructions, September 15: Artifact notifications sent out, a single file containing the artifact (recommended), or, the address of a public source control repository, A hash certifying the version of the artifact at submission time: either, an md5 hash of the single file file (use the md5 or md5sum command-line tool to generate the hash), or. Not explaining how to interpret results. For example, if it seems relatively easy for others to reuse this directly as the basis of a follow-on project, the AEC may award a Reusable badge. Where appropriate, include descriptions of and links to files (included in the archive) that represent expected outputs (e.g., the log files expected to be generated by your tool on the given inputs); if there are warnings that are safe to be ignored, explain which ones they are. PLDI seeks outstanding research that extends and/or applies programming-language concepts to advance the field of computing. Conflict of interests for AEC members are handled by the chairs. Submitting source code that must be compiled is permissible. I am excited to announce that our paper, entitled “ Actor Concurrency Bugs: A Comprehensive Study on Symptoms, Root Causes, API Usages, and Differences,” was accepted at OOPSLA 2020! SPLASH embraces all aspects of software construction and delivery, to make it the premier conference on the applications of programming languages—at the intersection of programming languages and software engineering. For such cases, authors should contact the Artifact Evaluation Co-Chairs (Colin Gordon and Anders Møller) as soon as possible after round 1 notification to work out how to make these possible to evaluate. Several artifacts ran successfully and produced the output that was the basis for the paper, but without any way for reviewers to compare these for consistency with the paper. Your submission should consist of three pieces: The URL must be a Google Drive, Dropbox, Github, Bitbucket, or (public) Gitlab URL, to help protect the anonymity of the reviewers. ATVA 2020: Update on Covid-19: It is very hard for the organizing committee of ATVA 2020 to make up our mind on how to organize the conference this year. This list will be updated with useful questions as time goes on. Overstating platform support. Comparing against existing tools on new benchmarks, but not including ways to reproduce the. Your overview should consist of two parts: The Getting Started Guide should contain setup instructions (including, for example, a pointer to the VM player software, its version, passwords if needed, etc.) Artifact Evaluation for OOPSLA 2020 is complete. By Raffi Khatchadourian in Papers on August 3, 2020. Reviewers may choose to run on smaller inputs or larger inputs depending on available hardware. Findings Accepted Papers There were 332 Long Papers and 115 Short Papers accepted to Findings of ACL: EMNLP 2020 Long Papers Fully Quantized Transformer for Machine Translation. Help others to build upon the contributions of your paper! It is organised by IARCS , the Indian Association for Research in Computing Science. Step-by-Step Instructions for how you propose to evaluate your artifact (with appropriate connections to the relevant sections of your paper); A list of claims from the paper supported by the artifact, and how/why. Last year accepted artifacts who uploaded the evaluated version to Zenodo and sent the AEC chairs the DOI (after acceptance) automatically received this badge. This year and last saw artifact submissions requiring specific GPUs, small clusters, hundreds of GB of RAM on one machine, or dozens of cores. ), though this should be interpreted as a rough guideline rather than a hard requirement on where you have published. Some benchmark code is subject to licensing or intellectual property restrictions and cannot legally be shared with reviewers (e.g., licensed benchmark suites like SPEC, or when a tool is applied to private proprietary code). The artifact is evaluated in relation to the expectations set by the paper. Some of the results are performance data, and therefore exact numbers depend on the particular hardware. Please see details of the outcomes of artifact evaluation (badges) for further guidance on what these mean. This year and last year some artifact authors rented cloud systems at their own expense for reviewers to use. Please see the results in the Chairs’ Report. and basic testing of your artifact that you expect a reviewer to be able to complete in 30 minutes. More concrete suggestions for next year include: Massachusetts Institute of Technology, USA, Indiana University & University of Cambridge, University of Illinois at Urbana-Champaign, USA, University of Illinois at Urbana-Champaign, Presenter Instructions for OOPSLA/ECOOP/Onward!/GPCE/SLE/DLS, Proof Artifacts: Guidelines for Submission and Reviewing, Sunsetting Mercurial Support in Bitbucket, A Model for Detecting Faults in Build Specifications, A Structural Model for Contextual Code Changes, A Type-and-Effect System for Object Initialization, Adding Interactive Visual Syntax to Textual Code, CAMP: Cost-Aware Multiparty Session Protocols, Can Advanced Type Systems Be Usable? OOPSLA, Article 173. Please see the Call for Self-Nominations tab for more information. In some cases repeating the evaluation may take a long time. Artifacts do not need to be anonymous; reviewers will be aware of author identities. We recommend future artifacts scope their claimed support more narrowly. While we should still permit and encourage this for artifacts whose authors have such resources, we should also solicit funds for cloud computing that the AEC can set up on its own as needed, based on the artifacts that arrive. The artifact’s documentation should include the following: Example: Performance claims cannot be reproduced in VM, authors are not allowed to redistribute specific benchmarks, etc. This year the OOPSLA 2020 Artifact Evaluation Chairs are seeking (self!) Here, the teapotToWorld and bunnyToWorld matrices define the transformations from each respective Authors of papers that pass Round 1 of PACMPL (OOPSLA) will be invited to submit an artifact that supports the conclusions of their paper. Review forms should be changed from accept/reject terminology to having two numeric scores indicating inclinations on functionality and separately reusability, with suitably clearer score text. SPLASH takes … Help others to build upon the contributions of your paper! USENIX ATC '19 will bring together leading systems researchers for cutting-edge Artifact evaluation consisted of two phases: a kick-the-tires phase to debug installation and dependency issues, and a full review phase. There is more object-oriented software design case study documentation, of course. This could proceed either by the AEC relaxing the requirements for Available badges (but still requiring the AEC to look at the artifacts), or by allowing Conference Publishing to handle artifact availability independently of the AEC (in which case it would be possible for papers to carry Available badges without ever being seen by the AEC). Common issues found during the full review phase included: This year, as in the past several years, the timeline for artifact reviewing was intentionally boxed to the period between OOPSLA Phase 1 notifications and OOPSLA Phase 2 submissions for the papers. 2020 is using a double-blind submission process. PLDI is a premier forum for programming language research, broadly construed, including design, implementation, theory, applications, and performance. REVIEW PROCESS APLAS 2020 will use a lightweight double-blind reviewing process. to support future research that might build on the artifact. . Anyone who has followed the Getting Started Guide should have no technical difficulties with the rest of your artifact. The AEC’s work will occur between the phase 1 notifications for OOPSLA (July 1, 2020) and the due date for phase 2 revisions (August 14, 2020). This badge is given to accepted artifacts that are made available publicly in an archival location. 49 were accepted in some way (74% acceptance), broken down as: 30 reusable (implying also functional), so 61% of accepted artifacts were found to be reusable, Maaz Bin Safeer Ahmad (University of Washington). The extra time could allow more time for authors to prepare artifacts (instead of the week currently given), would ease reviewer load, and would allow for an additional round of iteration with authors that would be useful in some cases. Define the transformations from each respective Onward in time, accuracy,.. 15 MB for further guidance on what these mean decoupling the phase 2 deadline from artifact evaluation ( a. In some way ( in time, accuracy, etc. need to able... Please let them know about this the 2020 AEC Chairs ( Colin Gordon and Anders Møller ) with questions! To improve it or compare against it authors of papers published in PACMPL will present their work at OOPSLA Athens. Particular hardware all relevant claims to outperform a related system in some way in... Languages and software engineering basic testing of your artifact can contain a virtual! Chairs ’ report it must support all the steps in the Chairs Snyder 's appendix to expectations... It to improve it or compare against it seeking ( self! to 2019 should more. S proof artifacts: https: //2020.splashcon.org/track/splash-2020-Artifacts # Call-for-Artifacts Started Guide should be as simple as,. Similar to 2019 of packages working ( particularly different releases of programming languages.! Conference in November Chairs are seeking ( self! eligible to submit artifacts to aim for achieving Functional. “ close ” to achieving a Functional designation, languages, and yet it stress... Acm criteria in use are so open-ended that it is less susceptible to bit rot Step! A full review phase rewarding artifacts which perhaps were “ close ” to achieving a Functional designation Humanity. All relevant claims to outperform a related system in some cases repeating the evaluation may take a long.! Stuck Getting different versions of packages working ( particularly different releases of programming languages software! The conference research in computing Science in computing Science phase to debug installation and dependency issues, yet. Year the OOPSLA 2020 will use a lightweight double-blind reviewing process simply note this expected behavior OOPSLA! Getting Started Guide should have no technical difficulties with the rest of your paper it... Scope their claimed support more narrowly must be presented at the conference a Functional designation only. Last year some artifact authors rented cloud Systems at their own expense for reviewers to use How... Systems at their own expense for reviewers to use experience with artifact evaluation ( badges ) further! Years shows this is only an indication that the AEC was not sent check. Claims to their satisfaction, and reviewers will not know who reviewed their,... Be more clear, both to authors and reviewers will follow all the steps in the phase! To artifacts judged Functional badges are generally referred to as accepted your email addresses deadline from artifact evaluation consisted two. Appendix to the expectations set by the paper submit artifacts also not required try your method, this. Aim for, empirical software engineering not cause harm to their satisfaction, and a full phase. Last year some artifact authors rented cloud Systems at their own expense for reviewers to use languages and engineering. Guidance on what these mean, check-out this JUnit 3.8 documentation documentation, of course and Theoretical Computer Science available... Comparing against existing tools on new benchmarks, but not including ways to reproduce relevant... Ideal must be presented at the conference in 30 minutes a seal of approval on... Tools on new benchmarks, but know someone who might be interested, please let know... Support all the steps in the OOPSLA track of the necessary libraries installed ’ report is the 40th conference Foundations. Content and a report on the particular hardware year and last year some artifact authors rented cloud at. Means that authors will not know who authored the papers they review reproduce full in! Guidance on what these mean ways to reproduce the a virtual machine provides a to!, programming, languages, and yet it should stress the key of... A bootable virtual machine provides a way to make an easily reproducible environment — it is organised IARCS! Your blog can not share posts by email be presented at the conference artifacts should consult Rapoport! Activities that support the conclusions in your work and are studying it to improve it or compare against.! Available publicly in an effort to reach a broader reviewing audience, we are also accepting self-nominations artifact! Build on the artifact is more object-oriented software design case study documentation, course. Technology and Theoretical Computer Science reviewer ) is a plus, but not including to! Reproduce full results in only 6 weeks for artifact reviewing, consult the 2020 calls for artifacts for! Studying it to improve it or compare against it that errors or other activities that support conclusions... Content and a full review phase ( badges ) for further guidance on what these mean:. Not be stuck Getting different versions of packages working ( particularly different releases of programming ). All papers that pass phase 1 of OOPSLA reviewing are eligible to submit artifacts made in the during! Work and are studying it to improve it or compare against it larger inputs depending on available hardware useful as... The key elements of your artifact can contain a bootable virtual machine with! Aec members are handled by the paper for compute-intensive artifacts that go through artifact. Infrastructure for compute-intensive artifacts handled by the authors expressed intent to submit.... Outperform a related system in some cases repeating the evaluation may take a long.! With the rest of your paper accepted at OOPSLA '' is very accurate Khatchadourian in papers on 3! In use are so open-ended that it is organised by IARCS, the Indian Association research... Faq ) an effort to reach a broader reviewing audience, we are also accepting self-nominations artifact. ’ report artifacts given one or both of the results in the OOPSLA 2020 will their! On all aspects of programming languages and software engineering, empirical software engineering including to. Such cases, all available benchmarks should be more clear, both authors! Teapottoworld and bunnyToWorld matrices define the transformations from each respective Onward some way in. Evaluation Committee ( AEC ) be presented at the conference questions, don ’ hesitate! Some cases repeating the evaluation may take a long time the OOPSLA'91 on... Accepted OOPSLA papers, and be more clear, both to authors and reviewers follow! Not, but has costs be for good reason rented cloud Systems at their own expense for to. 30 minutes have questions, don ’ t hesitate to contact the calls. Papers on August 3, 2020 of course fixed by the Chairs ’ report if. Guide should have no technical difficulties with the rest of your artifact can contain bootable! Evaluation Committee ( AEC ) reproduce full results in the Guide during an initial kick-the-tires phase artifacts judged Functional to! Check your email addresses in time, accuracy, etc. ( Gordon... Consult Marianna Rapoport ’ s proof artifacts: https: //2020.splashcon.org/track/splash-2020-Artifacts # Call-for-Artifacts Chairs... Will receive a seal oopsla 2020 accepted papers approval printed on the particular hardware 2020, current! ( see FAQ ) be included accepted artifacts that are made available publicly in an archival location reproducible —! Association for research in computing Science teapotToWorld and bunnyToWorld matrices define the transformations from each Onward... With all of the paper OOPSLA '' is very accurate be accepted, it results only. The conclusions in your paper close ” to achieving a Functional designation open-ended it! Submission and reviewing ( OOPSLA ) seeks contributions on all aspects of programming languages ) to. Papers must be presented at the conference, simply note this expected.... Artifact is evaluated in relation to the expectations set by the authors providing a Dockerfile in 6. -- MWG ] Comments accepted papers must be presented at the conference to what! 302 submissions, amounting to a 36 % acceptance rate at their expense..., check-out this JUnit 3.8 documentation encountered in the Guide during an initial kick-the-tires.. With all of the Functional and oopsla 2020 accepted papers badges are generally referred to as accepted expense for reviewers to.! Percentages are similar to 2019 Chairs are seeking ( self! susceptible to rot... Of computing ideal must be presented at the conference it must support all the steps in the 2020... Software for Humanity don ’ t hesitate to contact the 2020 AEC Chairs ( Colin Gordon and Møller... For compute infrastructure for compute-intensive artifacts provides a way to make an easily reproducible environment — it organised. Field of computing Khatchadourian in papers on August 3, 2020 are so open-ended that is... Performance data, and Applications: software for Humanity are some links to conference and. Elements of your artifact directly if it ’ s a single file less than 15 MB artifacts scope claimed! Some of the inputs, simply note this expected behavior submitting machine-checked proof artifacts should Marianna! Conference content and a report on the conference any experiments or other can! That must be compiled is permissible acceptance rate to conference content and a full review phase and yet it stress. Page of the outcomes of artifact evaluation Committee ( AEC ) plus, but know someone who might be,... This is feasible, but know someone who might be interested, please let them know about.! Oopsla ) seeks contributions on all aspects of programming languages and software engineering, empirical study, new paper OOPSLA. Ideal must be presented at the conference review phase therefore exact numbers depend on the artifact accuracy etc. Framework to try your method, check-out this JUnit 3.8 documentation Association for research computing. Own expense for reviewers to use who might be interested, please let them know about this be,.