IDA SDK C++ tricks and tips

[chants]

We see in graph.hpp the following events:

  grcode_user_title,          ///< render node title of a user-defined graph.
                              ///< \param g               (::mutable_graph_t *)
                              ///< \param node            (int)
                              ///< \param title_rect      (::rect_t *)
                              ///< \param title_bg_color  (int)
                              ///< \param dc              (HDC)
                              ///< \retval 0  did not render, ida will fill it with title_bg_color
                              ///< \retval 1  rendered node title

  grcode_user_draw,           ///< render node of a user-defined graph.
                              ///< NB: draw only on the specified DC and nowhere else!
                              ///< \param g          (::mutable_graph_t *)
                              ///< \param node       (int)
                              ///< \param node_rect  (::rect_t *)
                              ///< \param dc         (HDC)
                              ///< \retval 0  not rendered
                              ///< \retval 1  rendered

But HDC is certainly NOT what you thought it might be - a windows handle to a device context as IDA is totally portable. So after taking a stack trace, decompiling IDA64.exe and tracing the argument, its in fact a QT::QPainter object pointer. Basically the QT equivalent of a device context.

https://doc.qt.io/qt-5/qpainter.html

This is another totally undocumented, even if logical feature present. Unfortunately you must compile with QT headers, should link with the exact QT libs that IDA uses so it will reuse the QT DLL included with IDA, in this case for QTPainter - Qt5Gui.dll version 5.6.3.0

[bongos_man]

Originally Posted by chants

Unfortunately you must compile with QT headers, should link with the exact QT libs that IDA uses so it will reuse the QT DLL included with IDA, in this case for QTPainter - Qt5Gui.dll version 5.6.3.0

this is not entirely true. you can in fact substitute ida's qt dlls with your own (potentially modified) dlls. the primary difference between the stock qt dlls and those bundled with ida is that hex-rays builds theirs with "-qtnamespace QT" and thus all symbols with non-C linkage are not mangled identically to stock qt. here is the original post (from 2011, time flies ;]) from when idaq.exe first shipped:

https://www.hexblog.com/?p=250

not much has changed since then, here is info for ida 7.3 (i can confirm that this works, including complicated cases such as plugins loaded by qt itself--of course these too must be built against an appropriately configured qt installation):

https://www.hexblog.com/?p=1341

if you want to build qt without so much useless garbage, this e.g. will work:

..\qt-everywhere-opensource-src-5.6.3\configure -prefix C:/Qt/5.6.3.0 -debug-and-release -force-debug-info -platform win32-msvc2017 -opensource -confirm-license -shared -no-compile-examples -nomake examples -nomake tests -nomake tools -opengl desktop -no-warnings-are-errors -ltcg -no-rtti -sse2 -sse3 -ssse3 -sse4.1 -sse4.2 -avx -no-ssl -no-openssl -no-dbus -no-audio-backend -no-wmf-backend -no-qml-debug -no-native-gestures -mp -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcanvas3d -skip qtconnectivity -skip qtdeclarative -skip qtdoc -skip qtenginio -skip qtgraphicaleffects -skip qtimageformats -skip qtlocation -skip qtmacextras -skip qtmultimedia -skip qtquickcontrols -skip qtquickcontrols2 -skip qtscript -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtsvg -skip qttools -skip qttranslations -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns -qtnamespace QT

ida 7.2 (can also confirm my own success):

https://www.hexblog.com/?p=1261

see hexblog for more entries for previous versions, but again not much changes from release to release regarding qt.