Valgrind callgrind tools Part 3: Code coverage

In the Valgrind manual they list some tools you could write for Valgrind. One of the suggestions is to write a coverage tool. So a few months ago I sat down and did exactly that. It was a nice little tool and I happily posted it to the Valgrind developer mailing list only to discover that I was not the first person to do this nor was any of our solutions that good in the end. The easy part of the project is obtaining a recored of what was executed. In fact Cachegrind does a fantastic job of that. The only thing my tool did (that I thought was new) was automatically collect data on all the jumps which discovered after I wrote my tool was already present in the callgrind tool if you pass in the argument "--collect-jumps=yes". So me and several others have all written bad versions of callgrind and the cg_annotate script. Code coverage with Valgrind is made up of two problems. The first is obtaining the code execution and the second is applying this over the source for the program. The first is easy by using "callgrind --collect-jump=yes". The second part isn't as easy a new problem for every language, but at least for C++ using Roberto's parser makes this task a lot easier.

Here is an older output for a test that I wrote for QColumnView that shows some missing coverage:

valgrind --tool=callgrind --collect-jumps=yes ./tst_qcolumnview

./callgrind_coverage ~/dev/qt/tests/auto/qcolumnview/callgrind.out.16089 ~/dev/qt/src/gui/itemviews/qcolumnview.cpp

Function coverage - Has each function in the program been executed?
        Not executed: 612 void QColumnViewPrivate::_q_clicked(const QModelIndex &index)
        Coverage: 39/40 97%

Condition coverage - Has each evaluation point (such as a true/false decision) been executed?
443    for (int i = 0; i < ranges; ++i) {
842    for (; (i < list.count() && i < d->columns.count()); ++i) {
212    if (d->columns.isEmpty() || dx == 0)
976    if (!model() || !selectionModel())
976    if (!model() || !selectionModel())
229    if (!index.isValid() || d->columns.isEmpty())
229    if (!index.isValid() || d->columns.isEmpty())
272    if (leftEdge > -horizontalOffset()
336    switch (cursorAction) {
338        if (current.parent().isValid() && current.parent() != rootIndex())
338        if (current.parent().isValid() && current.parent() != rootIndex())
557        for (int i = columns.size() - 1; i >= 0; --i) {
549    while (currentColumn == -1 && parentIndex.isValid()) {
572    if (currentColumn == -1 && parent.isValid())
577    if (build && columns.size() > currentColumn + 1) {
580                             && !columns.at(currentColumn + 1)->rootIndex().isValid());
689    if (!columns.isEmpty() && columns.last()->isHidden())
689    if (!columns.isEmpty() && columns.last()->isHidden())
695    if (show && view->isHidden())
394    if (horizontalLength < viewportSize.width() && q->horizontalScrollBar()->value() == 0) {
813        if (!columns.isEmpty() && columns.last() == previewColumn)
1024    if (!model || columns.isEmpty())
1035            if (x != view->x() || viewportHeight != view->height())
1042            if (x != view->x() || viewportHeight != view->height())
368        if (diff < 0 && horizontalScrollBar()->isVisible()
877    if (currentParent == previous.parent()
877    if (currentParent == previous.parent()
879        for (int i = 0; i < d->columns.size(); ++i) {
893        for (int i = 0; i < d->columns.size(); ++i) {
512        if (!found && columns.at(i)->cornerWidget() == grip) {
        Coverage: 566/598 377%

Statement coverage - Has each line of the source code been executed?
        Coverage: 475/523 90%
-------------
Key:
 c == compiled
 e == executed
 r == return
 j == jump
 s == skipped
 - == comment/whitespace/not compiled
-------------
94      [j]  f: }
273     [c]:         && rightEdge <= ( -horizontalOffset() + viewport()->size().width())) {
286     [c]:             newScrollbarValue = rightEdge + horizontalOffset();
336     [j] f:     switch (cursorAction) {
369     [c]:             && horizontalScrollBar()->value() == horizontalScrollBar()->maximum()) {
579     [c]:         bool viewingChild = (!model->hasChildren(parent)
612     [c]: void QColumnViewPrivate::_q_clicked(const QModelIndex &index)
614     [c]:     Q_Q(QColumnView);
615     [c]:     QModelIndex parent = index.parent();
616     [c]:     QAbstractItemView *columnClicked = 0;
617     [c]:     for (int column = 0; column < columns.count(); ++column) {
618     [c]:         if (columns.at(column)->rootIndex() == parent) {
619     [c]:             columnClicked = columns[column];
623     [c]:     if (q->selectionModel() && columnClicked) {
624     [c]:         QItemSelectionModel::SelectionFlags flags = QItemSelectionModel::Current;
625     [c]:         if (columnClicked->selectionModel()->isSelected(index))
626     [c]:             flags |= QItemSelectionModel::Select;
627     [c]:         q->selectionModel()->setCurrentIndex(index, flags);
690     [c]:         columns.last()->setVisible(true);
726     [c]:         model()->fetchMore(index);
824     [c]:     previewColumn->setMinimumWidth(qMax(previewColumn->verticalScrollBar()->width(),
861     [rc]:     return list;
878     [c]:             && model()->hasChildren(current) && model()->hasChildren(previous)) {
939     [c]:             QItemSelectionModel *replacementSelectionModel =
941     [c]:             replacementSelectionModel->setCurrentIndex(
943     [c]:             replacementSelectionModel->select(
993     [c]:     QModelIndex br = model()->index(model()->rowCount(parent) - 1,
994     [c]:                                     model()->columnCount(parent) - 1,
1081    [c]:         opt.rect = QRect(option.rect.x() + option.rect.width() - width, option.rect.y(),

Note that this tool is a bit of a hack and is a work in progress so you will no doubt find issues. But it is good enough to be useful so I figured it would be time to publish:

The source for callgrind_coverage can be found in my callgrind tools git repository.