Showing posts with label cppcon_2021. Show all posts
Showing posts with label cppcon_2021. Show all posts

Jun 19, 2022

[CMake] Modern CMake in style

Reference:
Modern CMake Modules - Bret Brown - CppCon 2021
CppCon 2017: Mathieu Ropert “Using Modern CMake Patterns to Enforce a Good Modular Design”


$ clang++ main.cpp -o vsdmars -Wreturn-type
is same as:
target_compile_options(
    vsdmars PRIVATE
    $<$<COMPILE_LANG_AND_ID:CXX,Clang,GNU>:-Werror=return-type>
)

Find Modules

Loaded by the find_package()
Find<PackageName>.cmake // can have find_library(...) command


Use Modern CMake

  1. declare module with ADD_LIBRARY or ADD_EXECUTABLE
  2. declare build flags with TARGET_xxx()
  3. declare dependencies with TARGET_LINK_LIBRARIES
  4. Specify what is PUBLIC and what is PRIVATE
  5. Consider people are using your library through CMake


Don't

  1. breaking cmake target model
  2. breaking cmake features
    dev workflows
    functions
    standard modules
  3. making changes in source directories
  4. avoid version control operations
  5. toolchain details
    linkers
    compiler
  6. build requires magic flags like -DMAGIC_VAR
  7. Hijacking CMake variables
  8. Requiring special build targets
  9. Expecting certain environments
$ cmake && cmake --build && ctest && cmake --install


Write a module

name the file

vsdmars.cmake // include(vsdmars)
FindVsdmars.cmake // find_package(vsdmars REQUIRED)
vsdmarsConfig.cmake // find_package(vsdmars REQUIRED) <--- Most powerful

e.g.
vsdmarsConfig.cmake
function(target_needs_news)
    cmake_parse_arguments(parsed
        ""             #   options
        "TARGET"       # one-value keywords
        ""             # multi-value keywords
        ${ARGN}        # strings to parse
    )
    # TODO: defensive argument parsing
    set(target ${parsed_TARGET})
    message(VERBOSE "done, ${target}")
endfunction()


Defensive argument parsing

if(parsed_UNPARSED_ARGUMENTS)
    message(FATAL_ERROR
        "BAD ARGUMENT: ${parsed_UNPARSED_ARGUMENTS}"
    )
endif()

if(parsed_KEYWORDS_MISSING_VALUES)
    message(...)
endif()


Essential for CMake modules

  1. API documentation
    README
  2. Testing the cmake module we just wrote
    https://crascit.com/2016/10/18/test-fixtures-with-cmake-ctest/
  3. message() with verbosity
$ cmake --log-level=verbose


Installing a CMake modules

  1. share/cmake/vsdmars/vsdmarsConfig.cmake
  2. ship version files as well if needed


CMakeLists.txt for CMake modules

cmake_minimum_required(VERSION 3.21)
project(vsdmars LANGUAGES NONE) # if module is lang agnostic
enable_testing() # every CMakeLists.txt should have this

install(
    FILES vsdmarsConfig.cmake
    DESTINATION share/cmake/vsdmars
    COMPONENT vsdmars
)


Development Setup

CMAKE_GENERATOR=Ninja
CMAKE_TOOLCHAIN_FILE=path/to/ToolChain.cmake
CMAKE_BUILD_TYPE=Debug
DESTDIR=your/staging/dir


Development Workflow

$ git clone ...
$ cmake -B build -S vsdmars
$ cmake --build build
$ ctest --test-dir build
$ cmake --instal build \
    --prefix /opt/...
    --component vsdmars



Integration Workflow

$ git clone ...
$ cmake -B cxx-build \
    -S cxx-project \
    -DVSDMARS_DIR=./vsdmars/
$ cmake --build cxx-build
$ ctest --ctest-dir cxx-build
$ cmake --install cxx-build \
    --prefix /opt/vsdmars \
    --component cxx-project


Build flags don't scale

  • every change in public flags has to be propagated upwards
  • Most people give up and put every include director in a common/root build file.
  • Correct way of doing so is to put all 3rd party
    my/include/namespace
    into
    ./include/my/include/namespace


Modern build systems

  • Forbid/report circular and hidden dependencies
  • Help developer reason at module level
  • Do more than build as you are told


How?

  • Define your module build flags
  • Define your module dependencies
  • Keep out of other modules internals
  • Each module has a set of private flags
  • Each module has a set of public flags(required to build against its interface)
  • Build interfaces are transitive
  • Private means .cpp includes other headers
  • Public means .h includes other headers
  • Build flags are not gone but encapsulated
  • Can still with CPPFLAGS, CXXFLAGS and LDFLAGS, and external flags are not our concern anymore
  • Declare module with ADD_LIBRARY or ADD_EXECUTABLE
  • Declare build flags with TARGET_xxx()
  • Declare dependencies with TARGET_LINK_LIBRARIES
  • Specify what is PUBLIC and PRIVATE


Global setup

cmake_minimum_required(VERSION 3)
add_compile_options(-Wall -Werror etc.) // including flags effect ABI
add_library(vsdmarsLib src/xxx.cpp)



Declare flags

target_include_directories(vsdmarsLib PUBLIC include)
target_include_directories(vsdmarsLib PRIVATE src)

if (SOME_SETTING)
    target_compile_definitions(vsdmarsLIB
        PUBLIC WITH_SOME_SETTING
endif()


Declare dependencies

target_link_libraries(vsdmarsLib PUBLIC abc)
target_link_libraries(vsdmarsLib PRIVATE xyz)


Header only libraries

# nothing to build actually, just export headers during install/being pulled by other modules
add_library(vsdmarsLibHeaderOnly INTERFACE)
target_include_directories(vsdmarsLibHeaderOnly INTERFACE include)
target_link_libraries(vsdmarsLibHeaderOnly INTERFACE Boost::Boost)



Antipatterns



3rd party

cmake_minimum_required(VERSION 3.5)
find_package(GTest)
add_executable(foo ...)
target_link_libraries(foo GTest::GTest GTest::Main)
find_library(XXX_LIB xxx HINTS ${XXX_DIR}/lib)
add_library(xxx SHARED IMPORTED LOCATION ${XXX_LIB})
target_include_directories(xxx INTERFACE ${XXX_DIR}/include)
target_link_libraries(xxx INTERFACE Boost::boost)

Jun 13, 2022

[C++][CPPCON] s.t. about type kinds

Reference:
https://www.youtube.com/watch?v=va9I2qivBOA


std::variant
all_of
any_of
mismatch
equal
merge
set_union
set_intersection


Packs are a distinct kind

All types belong to a kind
 ... or 14(types), depending how you count
e.g.
nullptr
template names belong to another kind




There is a one-of-a-kind construct
Adding new kinds is almost unprecedented.

Packs are unlike everything else in C++
pointer to member function,
i.e.  ->*  .* return has no type.

std::integer_sequence


Hybrid algorithm, compile-time + runtime:
i.e. Linear search at compilation, binary search at runtime.

Jun 5, 2022

[C++][CPPCON 2021] s.t about Dynamically Loaded Libraries

Reference:
https://youtu.be/-dxCaM4GOqs

Definition:
Dynamic linking, opposed to static linking form a physical aspect of a program relocation is done at load time.


Dynamic loading ask for additional functionalities often involve library discovery relocation is done at run time maybe capable of 'unloading'.


in C++, object do not relocate but functions do.
Or to be precise, C/Assembly function kind do relocate, thus Golang could have dynamic growing stack size which relocate function at runtime.


After dlopen, unloading is often avoided due to this makes function having a 'lifetime'.
e.g. musl's dlcose is a noop. (complicating thread-local storage(TLS) implementation if library may unload)


Can objects from a library outlive the library?


Library lifetime realized in C++
- When loading a library, init. objects with static storage duration at namespace scope.
- When unloading a library, destruct objects with static storage duration.
- Loaded libraries are reference counted.


Interaction with thread_local
- When a thread starts, init. objects with thread storage duration at namespace scope.
- When a thread exits, destruct objects with thread storage duration.
- What happens if the library is unloaded before all threads exit?


glibc:
Do not unload the shared object during dlclose().
Consequently, the object's static and global variables are not reinitialized if the object is reloaded with dlopen() at a later time.


DF_1_NODELETE (elf.h)
- Set flag on the DSO until all thread_local objects defined in the DSO are destroyed
- After the flag being cleared, a subsequent dlclose() unloads the DSO
- i.e. dlclose() in the middle of destructing thread_local objects is a no-op


Unloading summary:
- Functions may have lifetime
- Implementations need to prevent objects with thread storage duration from outliving their destructors

May 7, 2022

[CPPCON 2021][notes] Debugging/Developing techniques

Reference:
https://www.youtube.com/watch?v=M7fV-eQwxrY

Define bugs

  • System is subject to a set of requirements.
  • A software defect is a non-conformity to requirements.
  • Pre-Curr-Post condition violated.
  • A non-conformity is a failure to meet one or more requirements.
  • A defect is incorrect program data causes a non-conformity.
  • A symptom is observable evidence of a defect.
  • A deterministic defect is a defect that does not change its symptoms under a well-defined set of conditions.
  • In contrast, a non-deterministic defect is a defect that changes its symptoms from run-to-run under a well-defined set of conditions.


Terminology

  • A context is the totality of the environment is which a program that exhibits symptoms is running
  • A problem report describes one or more symptoms in some context
  • Analogous context is a replica of the original context
  • Lab is the setting that we have total control over the context
  • Field is the setting that we have minimal or no control over the context


Relationship

Problem report -> Symptoms <-> Defects


Challenges

  • Problem report can be unhelpful (feed back from the user)
  • Problem report may not indicate actual problem
  • Collecting program state data may be difficult (log/setting/dump)
  • Symptoms may not indicate the cause
  • Defects and symptoms change as repair progresses
  • Fixing one defect may introduce new defects (messy design/quick fix)
  • Symptoms can be difficult to reproduce


Debugging process

Tend to think debugging is a linear process; i.e.
  • Characterize and reproduce
  • Locate
  • Classify
  • Understand
  • Repair


In reality tips

  • Review problem report
  • Characterize and reproduce problem
  • Clone if possible
  • Reproduce problem (loop)
    • understand problem
    • locate problem
    • classify problem
    • gain insight
    • attempt to repair
  • Problem fixed; deliver



In detail

Characterizing

  • Determining the context in which symptoms were observed
  • Version number, platform, resources allocated, external interfaces, configuration data, etc.
  • Information that allows you to instantiate an analogous context


Reproducing

  • Instantiating an analogous context, in the lab, or in the field
  • Running enough of the program/system to observe the reported symptoms
  • Developing new/updating existing test assets to demonstrate the failure
  • Make sure looking at the correct source code.
Characterizing and reproducing a problem is vital to the debugging process.


Understanding

Gaining ENOUGH knowledge about a problem and the surrounding code, that you believe you can make changes to carry out a repair.

At a minimum

  • located the incorrect lines of code
  • why the code is incorrect, root cause?
  • check the proposed classification
  • formulated a set of proposed changes
  • determine how the proposed changes could affect the runtime state

Inspect and verify the associated test assets

  • The test cases or harnesses may be broken
  • Test data should demonstrate correct and incorrect behavior

The defect may not be where you expect it

  • Keep an open mind and be ready to question all parts of the program

Ask yourself where the defect is not

  • trying to prove the absence of a defect reveals the defect

Explain to people why there is a defect, and why your proposed fix will resolve the defect

  • A local guru or bobblehead could be helpful - reach out for help if necessary


Locate the problem

Employ good development practices at the outset

  • Practice iterative, incremental, bottom-up development
  • Add functionality in small sections of code
  • Create test assets for each new increment of functionality
  • Verify that new code doesn't cause previous test cases to fail
  • Verify that new code passes its own test cases
  • Practice defensive programming

Alas

  • Well-written and extensive test assets
  • Preferably the whole product does this, at a minimum your fixes should
  • Adds runtime overhead, which can hinder the search for non-deterministic problems

Use trace logging

  • Generating output describing the program state during execution
  • In simpler cases, instrument code with print statements
  • In more complex systems, take advantage of existing logging facilities

Alas

  • Great way to stay 'on the path' when developing new code
  • An easy first step in narrowing down a problem's scope

Use debugging and analysis tools

  • Compiler warnings
  • Static code analysis tools (cppcheck, etc.)
  • Interactive debugger(gdb, lldb, udb, etc.)
  • Time-travel debugger(gdb, rr, udb, etc.)
  • Sanitizers (asan, tsan, ubsan, etc.)
  • Dynamic program analyzers(valgrind, etc.)
  • tracers (strace, wireshark, etc.)

Alas

  • for deterministic problems
  • not always useful for non-deterministic problem

Enable and/or add assertions

  • verify pre/curr/post condition of a function call.
  • verify expected program state

Alas


Use backtracking

  • Try to understand the program state at each backward step

Alas

  • Good for very simple programs/small search with deterministic problems

Divide and conquire (binary search)

  • Pick section of code to examine
  • Place an assertion or set a breakpoint.
  • Repeat until reveals the defect

Problem simplification

  • Gradually and strategically remove/comment out sections of irrelevant code

Alas

  • useful for debugging crashes of release builds
  • work backwards from the end of the section

Make the problem worse

  • Magnifies the problem signal.

Alas

  • helpful in first step finding and understanding the problem

Scientific method

  • Form a hypothesis consistent with observations
  • Implement tests to refute the hypothesis
  • If refuted, form a new hypothesis with new tests

Alas

  • time consuming; especially for code base that unfamiliar with
  • effective for all problems


Problem Types

Deterministic problems

  • Review the logs
  • Add assertions
  • Use interactive debugger

Non-deterministic problems

  • Review the logs
  • Create a debug build and see if it also exhibits the same symptoms
  • Add assertions where needed to verify invariants
  • Add assertions; comment out code, divide-and-conquer
  • Make the problem WORSE to magnify the problem
  • try low-overhead debugging tools "$gcc -g -o2"



Steps

Classifying

  • Determining a defect's category
  • Useful in formulating a repair strategy
  • Important information in subsequent reviews when considering preventive actions

Syntax errors
Syntax warnings
Implementation errors
Logic errors
Configuration errors


Repairing the problem

  • Implementing the appropriate fixed.
  • Passing the tests
  • Tests should be well written
  • Minimize changes to the system - keep changes small and localized
  • Verify repairs against test assets
    • All new/update tests should pass
    • All other tests should pass

Delivery

  • Practice good version control
  • Don't include fixes for more than one problem in one commit
  • Don't include extraneous changes (e.g. new features) in fix commits
  • Include new/update test assets in the fix commits
  • Write commit comments clear and concise

Verify tests again

  • Double check all new/update tests pass
  • Double check all other tests pass

Create documentation for posterity

  • How the defect was noticed
  • The conditions under which the defect occurred - the context
  • Steps necessary to reproduce the defect - the analogous context
  • Techniques and tools used to localize the defect
  • Defect's category
  • Underlying root cause of the defect
  • Latent defects precluded by fixing this defect
  • Possible latent defects left unaddressed
  • Mistake made and recommendations for preventive actions



Developing new feature

  • Practice defensive programming
    • Assume the worst case could happen at any time.
  • Employ an appropriate iterative and incremental development process
    • Decide what needs to be achieved
    • Formulate a plan for the achievement
    • Understand the invariants, requirements, and context, then design the solution.
    • Implement the solution in small, discrete, testable chunks
    • Write code to verify invariants, pre-cur-post conditions and self-test complex components
  • Consider employing the principles of test-driven design
  • Employ good configuration management practice EVERYWHERE.

Feb 12, 2022

[C++][notes] Branchless Programming in C++ - Fedor Pikus - CppCon 2021

Reference:
Branchless Programming in C++ - Fedor Pikus - CppCon 2021
Computer Architecture - A quantitative approach (Hennessy, Patterson) Appendix-C 
https://vsdmars.blogspot.com/2016/01/likely-or-unlikely-easy-misleading.html


What determines performance?

  • Optimal algorithm
    Get the result with minimal work.
  • Efficient use of language
    do not do any unnecessary work
  • Efficient use of hardware
    use all available resources
    at the same time
    all the time


Hazards

  • Structural; use stall/bubble
  • Data; forwarding, stall/bubble in the middle of pipeline
  • Branch;
    Freeze/flush; holding or deleting any instructions after the branch until the branch destination is known.
    Treat every branch as not taken.
    Treat every branch as taken.
    Delayed branch.
As for branch hazards, we usually use static branch prediction by profiling.
A branch is usually bimodally distributed.
As for dynamic branch prediction; we use branch history table.


Tools

Google benchmark is our friend https://github.com/google/benchmark
$ perf stat our_binary  // shows branch-misses

When do benchmark like this(branch mis-predicting), we should avoid the predicting is done by the compiler, which is damn smart to generate efficient code.
  • Optimizing away branches almost always results in doing more work.
  • CPU usually has idle compute resources which it can handle a bit of extra work.
  • Branch mis-prediction is very expensive.
  • Trade off between the extra work vs. the code of the branch is usually impossible to predict; must be measured.


Less branch means better

Tricks:
use hashmap[] as branches. hashmap is O(1).
However; keep in mind this cause extra memory thus use iff branch is poorly predicted and the extra hashmap computations are small.
  • Sometimes the compiler will do a branchless transformation for you. (conditional move instruction)
  • Compiler's branchless optimization is usually better than ours'.
  • This is almost always branchless in reality:
    return cond ? x : y;
  • Never optimize code preemptively.
  • Optimize only if the profiler shows high mis-prediction rate.
  • Optimizations depend on the compiler.
 if (b) s += x;
 //vs.
 s += b * x;
  • Sometimes branchless code is not really branchless
  • Indirect function calls are similar to branches
    if (cond) f1(); else f2();
  • Can be converted to branchess:
    funcptr f[2] = {&f2, &f1};
    (f[cond])();
  • Above code almost never works.
    If f1 and f2 were inlined..
  • Always measure


Lessons learned

  • Predicted branches are cheap
  • Mis-predicted branches are very expensive - pipeline flush
  • Optimization - user fewer(or zero) branches
  • Always use profiler to detect and validate optimization locations
  • Don't fight with the compiler - sometimes it does the job for you