Code Submission and Repositories
Problems: FPGA conferences receive and publish papers about great applications that push the state-of-the-art, but these seldom become available as benchmarks and building blocks. Making an application or design usable outside of the research group that developed it is hard work that is not directly praised and rewarded by our current system. Current archives are not setup to capture much beyond PDF documents. (ACM DL does collect Supplemental and Auxiliary Material; that presents an opportunity we should consider how to use.)
Vision: With cheap storage space and rich network connectivity, it should now be possible to move beyond solely using text and picture representations of research contributions (traditional papers). We should be able to include actual design implementations in the artifacts submitted for review and preserved in our archival repositories. At some day in the future, we should be able to download an implementation to go with (most of) the papers in our conferences and journals.
We certainly see individual groups that step up to releasing designs and code on their web sites. How do we build on the example of these pioneers so that this becomes more the rule than the exception?
Questions
- What does this look like?
How do we get there? (e.g. new venues? special sections or tracks in existing venues?)
- The specific issue of FPGA designs is the most unique to this community, but there is also merit in in supporting mapping tools as well. Should they be separated? What are the different issues?
Pioneering Example
See (and contribute to) a list of PioneeringExamples.
Challenges to Address
- Setting standards for submissions and archiving?
Dealing with diversity of tools and languages (e.g. Verilog, VHDL, C, TDF, ...), perhaps including proprietary? (e.g. Bluespec System Verilog)
- How make reviewing viable?
- Maybe citation count isn't the best way to quantify goodness and contribution?
Are there other metrics to capture to reflect the value of this contribution? (e.g. downloads, design-in's)
- How can industry experts (who don't typically publish papers about their use of a design) give feedback on valuable, contributed designs from academics?
Action
We are forming a TCFPGA Action Committee to refine vision, further identify key challenges, and figure out the steps we can take to move forward toward this vision. We can collect working thoughts here on this wiki. Contact andre@seas.upenn.edu if you'd like to contribute to the discussion.