Content deleted Content added
Harrygoods76 (talk | contribs) Linked WP page |
Jnestorius (talk | contribs) |
||
| (9 intermediate revisions by 8 users not shown) | |||
Line 1:
{{short description|Automatic repair of software bugs}}
'''Automatic bug-fixing''' is the automatic [[Patch (computing)|repair]] of [[software bug]]s without the intervention of a human programmer.<ref>{{Cite journal |last=Rinard |first=Martin C. |year=2008 |title=Technical perspective ''Patching'' program errors |journal=Communications of the ACM |volume=51 |issue=12 |pages=86 |doi=10.1145/1409360.1409381 |s2cid=28629846}}</ref><ref>{{Cite journal |last=Harman |first=Mark |year=2010 |title=Automated patching techniques |journal=Communications of the ACM |volume=53 |issue=5 |pages=108 |doi=10.1145/1735223.1735248 |s2cid=9729944}}</ref><ref name="Gazzola2019">{{Cite journal |last1=Gazzola |first1=Luca |last2=Micucci |first2=Daniela |last3=Mariani |first3=Leonardo |year=2019 |title=Automatic Software Repair: A Survey |url=https://boa.unimib.it/bitstream/10281/184798/2/08089448_final.pdf |journal=IEEE Transactions on Software Engineering |volume=45 |issue=1 |pages=34–67 |doi=10.1109/TSE.2017.2755013 |hdl=10281/184798 |s2cid=57764123|doi-access=free }}</ref> It is also commonly referred to as ''automatic patch generation'', ''automatic bug repair'', or ''automatic program repair''.<ref name="Gazzola2019">{{Cite journal |last1=Gazzola |first1=Luca |last2=Micucci |first2=Daniela |last3=Mariani |first3=Leonardo |year=2019 |title=Automatic Software Repair: A Survey |url=https://boa.unimib.it/bitstream/10281/184798/2/08089448_final.pdf |journal=IEEE Transactions on Software Engineering |volume=45 |issue=1 |pages=34–67 |doi=10.1109/TSE.2017.2755013 |hdl=10281/184798 |s2cid=57764123|doi-access=free }}</ref> The typical goal of such techniques is to automatically generate correct [[Patch (computing)|patches]] to eliminate bugs in [[software program]]s without causing [[software regression]].<ref>{{Cite book |last1=Tan |first1=Shin Hwei |title=2015 IEEE/ACM 37th IEEE International Conference on Software Engineering |last2=Roychoudhury |first2=Abhik |date=2015 |publisher=IEEE |isbn=978-1-4799-1934-5 |pages=471–482 |chapter=relifix: Automated repair of software regressions |doi=10.1109/ICSE.2015.65 |s2cid=17125466}}</ref>
== Specification ==
Automatic bug fixing is made according to a specification of the expected behavior which can be for instance a [[formal specification]] or a [[test suite]].<ref name="genprog2009">{{Cite book |last1=Weimer |first1=Westley |title=Proceedings of the 31st International Conference on Software Engineering |last2=Nguyen |first2=ThanhVu |last3=Le Goues |first3=Claire|author3-link=Claire Le Goues |last4=Forrest |first4=Stephanie |date=2009 |publisher=IEEE |isbn=978-1-4244-3453-4 |pages=364–374 |chapter=Automatically finding patches using genetic programming |citeseerx=10.1.1.147.8995 |doi=10.1109/ICSE.2009.5070536 |s2cid=1706697}}</ref>
A test-suite – the input/output pairs specify the functionality of the program, possibly captured in [[Assertion (software development)|assertions]] can be used as a [[test oracle]] to drive the search. This oracle can in fact be divided between the ''bug oracle'' that exposes the faulty behavior, and the ''regression oracle'', which encapsulates the functionality any program repair method must preserve. Note that a test suite is typically incomplete and does not cover all possible cases. Therefore, it is often possible for a validated patch to produce expected outputs for all inputs in the test suite but incorrect outputs for other inputs.<ref name="kali">{{Cite book |last1=Qi |first1=Zichao |title=Proceedings of the 2015 International Symposium on Software Testing and Analysis |last2=Long |first2=Fan |last3=Achour |first3=Sara |last4=Rinard |first4=Martin |date=2015 |publisher=ACM |isbn=978-1-4503-3620-8 |chapter=An
Another way to specify the expected behavior is to use [[formal specification]]s<ref name="autofixe">{{Cite journal |last1=Pei |first1=Yu |last2=Furia |first2=Carlo A. |last3=Nordio |first3=Martin |last4=Wei |first4=Yi |last5=Meyer |first5=Bertrand |last6=Zeller |first6=Andreas |date=May 2014 |title=Automated Fixing of Programs with Contracts |journal=IEEE Transactions on Software Engineering |volume=40 |issue=5 |pages=427–449 |arxiv=1403.1117 |bibcode=2014arXiv1403.1117P |doi=10.1109/TSE.2014.2312918 |s2cid=53302638}}</ref><ref>{{Cite journal |title=Contract-based Data Structure Repair Using Alloy |citeseerx=10.1.1.182.4390}}</ref> Verification against full specifications that specify the whole program behavior including functionalities is less common because such specifications are typically not available in practice and the computation cost of such [[formal verification|verification]] is prohibitive. For specific classes of errors, however, implicit partial specifications are often available. For example, there are targeted bug-fixing techniques validating that the patched program can no longer trigger overflow errors in the same execution path.<ref name="codephage" />
Line 14:
=== Generate-and-validate ===
Generate-and-validate approaches compile and test each candidate patch to collect all validated patches that produce expected outputs for all inputs in the test suite.<ref name="genprog2009" /><ref name="kali" /> Such a technique typically starts with a test suite of the program, i.e., a set of [[test
<!-- mutation based repair -->
One way to generate candidate patches is to apply [[program mutation|mutation operators]] on the original program. Mutation operators manipulate the original program, potentially via its [[abstract syntax tree]] representation, or a more coarse-grained representation such as operating at the [[Statement (programming)|statement]]-level or [[Block (programming)|block]]-level. Earlier [[Genetic improvement (computer science)|genetic improvement]] approaches operate at the statement level and carry out simple delete/replace operations such as deleting an existing statement or replacing an existing statement with another statement in the same source file.<ref name=genprog2009 /><ref name="genprog2012">{{Cite book |last1=Le Goues |first1=Claire|author1-link=Claire Le Goues |title=2012 34th International Conference on Software Engineering (ICSE) |last2=Dewey-Vogt |first2=Michael |last3=Forrest |first3=Stephanie |last4=Weimer |first4=Westley |date=2012 |publisher=IEEE |isbn=978-1-4673-1067-3 |pages=3–13 |chapter=A Systematic Study of Automated Program Repair: Fixing 55 out of 105 Bugs for $8 Each |citeseerx=10.1.1.661.9690 |doi=10.1109/ICSE.2012.6227211 |s2cid=10987936}}</ref> Recent approaches use more fine-grained operators at the [[abstract syntax tree]] level to generate more diverse set of candidate patches.<ref name=spr /> Notably, the statement deletion mutation operator, and more generally removing code, is a reasonable repair strategy, or at least a good fault localization strategy.<ref>{{Cite book |last1=Qi |first1=Zichao |last2=Long |first2=Fan |last3=Achour |first3=Sara |last4=Rinard |first4=Martin |title=Proceedings of the 2015 International Symposium on Software Testing and Analysis |chapter=An analysis of patch plausibility and correctness for generate-and-validate patch generation systems |date=2015-07-13 |chapter-url=http://dx.doi.org/10.1145/2771783.2771791 |pages=24–36 |___location=New York, NY, USA |publisher=ACM |doi=10.1145/2771783.2771791|hdl=1721.1/101586 |isbn=9781450336208 |s2cid=6845282 }}</ref>
<!-- fix templates -->
Line 30:
SemFix<ref name="semfix" /> uses component-based synthesis.<ref>{{Cite book |last1=Jha |first1=Susmit |url=http://techreports.lib.berkeley.edu/accessPages/EECS-2010-15.html |title=Oracle-guided component-based program synthesis |last2=Gulwani |first2=Sumit |last3=Seshia |first3=Sanjit A. |last4=Tiwari |first4=Ashish |date=2010-05-01 |publisher=ACM |isbn=9781605587196 |pages=215–224 |doi=10.1145/1806799.1806833 |s2cid=6344783}}</ref>
Dynamoth uses dynamic synthesis.<ref>{{Cite book |last1=Galenson |first1=Joel |title=CodeHint: dynamic and interactive synthesis of code snippets |last2=Reames |first2=Philip |last3=Bodik |first3=Rastislav |last4=Hartmann |first4=Björn |last5=Sen |first5=Koushik |date=2014-05-31 |publisher=ACM |isbn=9781450327565 |pages=653–663 |doi=10.1145/2568225.2568250 |s2cid=10656182}}</ref>
S3<ref>{{Cite book |last1=Le |first1=Xuan-Bach D. |url=https://ink.library.smu.edu.sg/sis_research/3917 |title=Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering - ESEC/FSE 2017 |last2=Chu |first2=Duc-Hiep |last3=Lo |first3=David |last4=Le Goues |first4=Claire |author4-link=Claire Le Goues|last5=Visser |first5=Willem |date=2017-08-21 |publisher=ACM |isbn=9781450351058 |pages=593–604 |doi=10.1145/3106237.3106309 |s2cid=1503790}}</ref> is based on [[syntax-guided synthesis]].<ref>{{Cite book |last1=Alur |first1=Rajeev |title=2013 Formal Methods in Computer-Aided Design |last2=Bodik |first2=Rastislav |last3=Juniwal |first3=Garvit |last4=Martin |first4=Milo M. K. |last5=Raghothaman |first5=Mukund |last6=Seshia |first6=Sanjit A. |last7=Singh |first7=Rishabh |last8=Solar-Lezama |first8=Armando |last9=Torlak |first9=Emina |author9-link=Emina Torlak|year=2013 |isbn=9780983567837 |pages=1–8 |chapter=Syntax-guided synthesis |citeseerx=10.1.1.377.2829 |doi=10.1109/fmcad.2013.6679385 |last10=Udupa |first10=Abhishek}}</ref>
SearchRepair<ref name="searchrepair">{{Cite book |last1=Ke |first1=Yalin |title=Proceedings of the 2015 30th IEEE/ACM International Conference on Automated Software Engineering |last2=Stolee |first2=Kathryn |last3=Le Goues |first3=Claire |author3-link=Claire Le Goues|last4=Brun |first4=Yuriy |date=2015 |publisher=ACM |isbn=978-1-5090-0025-8 |series=ASE 2015 |___location=Lincoln, Nebraska |pages=295–306 |chapter=Repairing Programs with Semantic Code Search |doi=10.1109/ASE.2015.60 |s2cid=16361458}}</ref> converts potential patches into an SMT formula and queries candidate patches that allow the patched program to pass all supplied test cases.
=== Data-driven ===
Line 61:
== Overfitting ==
Sometimes, in test-suite based program repair, tools generate patches that pass the test suite, yet are actually incorrect, this is known as the "overfitting" problem.<ref name="overfitting">{{Cite book |last1=Smith |first1=Edward K. |title=Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering |last2=Barr |first2=Earl T. |last3=Le Goues |first3=Claire|author3-link=Claire Le Goues |last4=Brun |first4=Yuriy |date=2015 |publisher=ACM |isbn=978-1-4503-3675-8 |series=ESEC/FSE 2015 |___location=New York, New York |pages=532–543 |chapter=Is the Cure Worse Than the Disease? Overfitting in Automated Program Repair |doi=10.1145/2786805.2786825 |s2cid=6300790}}</ref> "Overfitting" in this context refers to the fact that the patch overfits to the test inputs. There are different kinds of overfitting: incomplete fixing means that only some buggy inputs are fixed, regression introduction means some previously working features are broken after the patch (because they were poorly tested). Early prototypes for automatic repair suffered a lot from overfitting: on the Manybugs C benchmark, Qi et al.<ref name="kali" /> reported that 104/110 of plausible GenProg patches were overfitting. In the context of synthesis-based repair, Le et al.<ref>{{Cite journal |last1=Le |first1=Xuan Bach D. |last2=Thung |first2=Ferdian |last3=Lo |first3=David |last4=Goues |first4=Claire Le |date=2018-03-02 |title=Overfitting in semantics-based automated program repair |url=https://ink.library.smu.edu.sg/sis_research/3986 |journal=Empirical Software Engineering |language=en |volume=23 |issue=5 |pages=3007–3033 |doi=10.1007/s10664-017-9577-2 |issn=1382-3256 |s2cid=3635768}}</ref> obtained more than 80% of overfitting patches.
One way to avoid overfitting is to filter out the generated patches. This can be done based on dynamic analysis.<ref>{{Cite book|last1=Xin|first1=Qi|last2=Reiss|first2=Steven P.|title=Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis |chapter=Identifying test-suite-overfitted patches through test case generation |date=2017-07-10|chapter-url=http://dx.doi.org/10.1145/3092703.3092718|pages=226–236|___location=New York, NY, USA|publisher=ACM|doi=10.1145/3092703.3092718|isbn=978-1-4503-5076-1|s2cid=20562134}}</ref>
Alternatively, Tian et al. propose heuristic approaches to assess patch correctness.
== Limitations of automatic bug-fixing ==
Line 79:
In C, the Manybugs benchmark collected by GenProg authors contains 69 real world defects and it is widely used to evaluate many other bug-fixing tools for C.<ref name=genprog2012 /><ref name=prophet /><ref name=spr /><ref name=angelix />
In [[Java (programming language)|Java]], the main benchmark is Defects4J now extensively used in most research papers on program repair for Java.<ref name="capgen">{{Cite book |last1=Wen |first1=Ming |last2=Chen |first2=Junjie |last3=Wu |first3=Rongxin |last4=Hao |first4=Dan |last5=Cheung |first5=Shing-Chi |title=Proceedings of the 40th International Conference on Software Engineering |chapter=Context-aware patch generation for better automated program repair |date=2018 |___location=New York, New York, USA |publisher=ACM Press |pages=1–11 |doi=10.1145/3180155.3180233 |isbn=9781450356381 |s2cid=3374770|url=https://repository.hkust.edu.hk/ir/Record/1783.1-92186 |chapter-url=
== Example tools ==
Line 93:
* LeakFix:<ref name=leakfix /> A tool that automatically fixes memory leaks in C programs.
* Prophet:<ref name=prophet /> The first generate-and-validate tool that uses machine learning techniques to learn useful knowledge from past human patches to recognize correct patches. It is evaluated on the same benchmark as GenProg and generate correct patches (i.e., equivalent to human patches) for 18 out of 69 cases.<ref name=prophet />
* SearchRepair:<ref name=searchrepair /> A tool for replacing buggy code using snippets of code from elsewhere. It is evaluated on the IntroClass benchmark<ref name="introclassmanybugs">{{Cite journal |last1=Le Goues |first1=Claire|author1-link=Claire Le Goues |last2=Holtschulte |first2=Neal |last3=Smith |first3=Edward |last4=Brun |first4=Yuriy |last5=Devanbu |first5=Premkumar |last6=Forrest |first6=Stephanie |last7=Weimer |first7=Westley |date=2015 |title=The Many ''Bugs'' and Intro ''Class'' Benchmarks for Automated Repair of C Programs |journal=IEEE Transactions on Software Engineering |volume=41 |issue=12 |pages=1236–1256 |doi=10.1109/TSE.2015.2454513 |doi-access=free}}</ref> and generates much higher quality patches on that benchmark than GenProg, RSRepair, and AE.
* Angelix:<ref name=angelix /> An improved solver-based bug-fixing tool. It is evaluated on the GenProg benchmark. For 10 out of the 69 cases, it generate patches that is equivalent to human patches.
* Learn2Fix:<ref name=learn2fix /> The first human-in-the-loop semi-automatic repair tool. Extends GenProg to learn the condition under which a semantic bug is observed by systematic queries to the user who is reporting the bug. Only works for programs that take and produce integers.
Line 120:
== External links ==
* {{URL|
[[Category:Debugging]]
| |||