Showing posts with label study_note. Show all posts
Showing posts with label study_note. Show all posts

Mar 23, 2019

[C++] Life is short, learn C++ in the smart way

Life is short, learn C++ in the smart way.
  1. Assembly, Programming from the Ground Up, by Jonathan Bartlett
  2. Linker, Linkers and Loaders, by John R. Levine
  3. Linker, Advanced C and C++ Compiling, by Milan Stevanovic
  4. Linking articles, https://eli.thegreenplace.net/tag/linkers-and-loaders by Eli Bendersky
  5. C Programming Language, by Brian W. Kernighan, Dennis M. Ritchie
  6. Inside the C++ Object Model, by Stanley B. Lippman


That's it, alas, understanding what's name mangling is essential,
and why it's essential; last, always have https://godbolt.org/ opened.

What about r-value/l-value, lambda expression, template programming, concurrency, thread,
memory model, ODR, RAII, SFINAE, CTAD, RTTI, etc. ?

It could be fathom easily by reasoning after above readings.

Actually, after above readings, nearly all programming languages are easy to fathom by reasoning :-)

Nov 4, 2018

[paper] RobinHood: Tail Latency Aware Caching -- Dynamic Reallocation from Cache-Rich to Cache-Poor

https://www.usenix.org/conference/osdi18/presentation/berger


Even virtual memory can be considered as 'resource' thus managing it with a
'scheduler'.

Sep 6, 2018

[recap math] Linear algebra, Calculus, Statistics ~ [2018]

Calculus:
progress


LA:
progress


Statistics:
progress



Courses:
https://www.cs.cmu.edu/bs-in-artificial-intelligence/curriculum

https://www.coursera.org/learn/machine-learning

https://www.coursera.org/specializations/deep-learning

https://www.coursera.org/learn/convolutional-neural-networks

---
https://www.coursera.org/learn/digital

https://www.coursera.org/learn/image-processing

https://www.coursera.org/specializations/probabilistic-graphical-models


Algorithms for Convex Optimization:
https://nisheethvishnoi.wordpress.com/convex-optimization/

May 13, 2018

[ACCU 2018] 105 STL Algorithms in Less Than an Hour - Jonathan Boccara

105 STL Algorithms in Less Than an Hour - Jonathan Boccara [ACCU 2018]

http://en.cppreference.com/w/cpp/algorithm

Why algorithms?

  • off-by-one error
  • empty loops
  • naive complexity

STL algorithms can make code simpler

  • Replace for-loops by the right algorithm

Improve code expressiveity

  • Understand common vocabulary
  • Write in the common vocabulary    

Understand technical aspects

  • complexity
  • pre/post requisites
  • look into the implementation    

Write our own algorithms

  • Combine existing algorithms together
  • Enrich a family
  • Start a new family

May 12, 2018

[design][microservice] Microservices at Netflix scale

Watch note for Microservices at Netflix scale

Microservices

What's the cost?

Make assumptions

Then we could prioritize which is important.

Services should be stateless

  • Not rely on sticky session
  • Chaos testing
  • Verify stateless (kill the instance randomly)

Scale out vs. scale up

  • NoSQL at scale (Cassandra, Scylla)
  • CAP available
    (Local quorum vs. async copy to remote region)

Redundancy and Isolation for resiliency

  • Avoid single point of failure
  • Make more than one of _anything_
  • Isolate the blast radius for any given failure

Destructive testing

    Check check check

Billing services


Arch:
Request Cache!


Consider maintenance efforts

Writing tools for automation, less human involve would be better!

Reliability

Cascading failures
Tools to check microservices failure
Failure has grades. Some low grade failure is OK with backup services.
Tools: Hystrix, or Envoy




Containers



[design][microservice] Known Before Scaling


Watch note for What I Wish I Had Known Before Scaling Uber to 1000 Services • Matt Ranney


Microservices


  • immutable?
  • append only?


Move and release independently
Own your uptime
Use the best tool for the job

What are the costs?
Everything is RPC
What if it breaks?

Everything is a trade off.

[design][microservice] The hardest part of microservices is your data

Watch note for The hardest part of microservices is your data


slides:
https://www.slideshare.net/ceposta/the-hardest-part-of-microservices-your-data
  • Microservices is about optimizing for SPEED.
  • Manage (aka. reduce) dependencies.
  • Data is a major dependency.
  • Focus on domain models, not data models
  • Stick with these conveniences as long as you can.
  • A microservice has its own database
  • We need to understand something about the data inside our services and the data outside our services
    • Thus, we are building a distributed system, thus, beware of 
      • network latency, 
      • network partition
  • Plan for failures.
    • Build concepts of time, delay, network, and failures into the design as a first-class citizen
  • How do you “read” data and how do you “update” data.
  • Performance: N+1 Query Problem
  • For our reads and writes, we need some "consistency"
  • We need reads and writes. But we expect failures. 
    • CAP tells us to pick 2: Consistency, Availability, Partition Tolerance
    • CAP is a bad way to think about this.
  • Consistency model
    https://en.wikipedia.org/wiki/Consistency_model
  • What consistency model do you need, depending on what role you’re playing?

[design] Serverless Architecture (FaaS)

Watching note for Serverless: the Future of Software Architecture • Peter Sbarski

cross reference:
Designing distributed system

Function as Service (execute custom code)
Backend as a Service (3rd-party services)

Adopt

  • Pipelines as code
  • APIs as a product
  • Decoupling secret management from source code
  • Hosting PII data in the EU
  • Legacy in a box
  • Lightweight architecture decision records
  • Progressive web applications
  • Prototyping with invision and sketch
  • Serverless architecture

e.g
AWS Lambda for compute
S3 for storage


Principles of Serverless Architecture

  • Use a code-execution compute service to run code on demand
  • Write single-purpose statelesss functions
  • Design push-based, event-driver pipelines
  • Create thicker, more powerful front ends
  • Embrace third party services

Use a code-execution compute service to run code on demand

Do not run on a server.
Focus on function, not application update/upgrade
(e.g patch Apache web server etc.)

Compute as backend:
API-Gateway is needed for REST API


Compute as glue:
Act as Pipeline


Write single-purpose statelesss functions

Pure function
Only 0 or 1 transformation

Design push-based, event-driver pipelines

One event triggers another event.
e.g




e.g.
Use websocket, push based.

Create thicker, more powerful front ends

traditional way:


Changed way:


Embrace third party services

Benefits

  • time to market
  • scale effortlessly (deal with spikes traffic)
  • disruptive cost model
  • no more server to manage
  • versatile
  • lower cost
  • less code
  • easy to scale and flexible

Cons

  • Not for everyone
  • Service level and customization
  • Vendor lock-in
  • Decentralization

Architecture

Go for microservices
Use API-Gateway
https://www.algolia.com/ as search service

[design] Event-driven system pattern

Watch note for The Many Meanings of Event-Driven Architecture • Martin Fowler

4 event driven patterns

Event notification

    啊就是類似 GUI  一個thread在while loop 聽event...
  • Decouple receiver from sender

Event-carried state transfer

    啊就是將event存成state 傳遞...
  • Decoupling
  • Reduced load on supplier 

Event sourcing

    啊就是 log 咩 RAFT it is...
  • Create event object and persist the object
  • Process the persisted object
  • No state, can restore the state machine from log    (like git(or other version application) system is using event sourcing)
can be used in:
  • audit
  • debugging
  • historic state
  • alternative state
  • memory image        

Command Query Responsibility Segregation(CQRS)

    啊就是 rwlock啊...
  • Seperate read/write into two application.
  • Write: Command model
  • Read: Query model

Reference:

[design] 4 patterns for distributed sysmtem architecture

4 patterns for distributed sysmtem architecture

Modern three-tier

Strengths

  • Rich front-end framework
  • Hip, scalable middle tier
  • Basically infinitely scalable data tier

Weaknesses

  • State in the middle tier


Sharded

Strengths

  • Client isolation is easy (data and deployment)
  • Known, simple technologies

Weaknesses

  • Complexity
  • No comprehensive view of data
  • Oversized shards

DB:
read replicate


  • Partition problem.
    • Master is dead, reelect master.

Lambda

Streaming vs. Batch
Unbounded(immutible data) vs. Bounded

e.g
event system

Strengths

  • Optimizes subsystems based on operational requirements
  • Good at unbounded data

Weaknesses

  • Complex to operate and maintain
    • write same code twice
  • For analysis for best!

Streaming

  • Integration is a first-class concern
  • Life is dynamic; databases are static
  • Tables are streams and streams are tables
  • Keep your services close, your computation closer

Integration

Bad:

Kafka:


Event stream:
transfer database/table to stream...
i.e into a log!

Storing data in messages
First-class message system
  • events get consumed though message stream system
  • Then, each request/response from the sub-system which hooked on to the stream system.
    i.e stream system act like the 'HUB'
  • Keep stream computation near our code.
Winner is: Streaming :-D


May 10, 2018

[design] single sign on (SSO) arch


[design] thinking process

A strong process is crucial to successfully solving system design questions.

4 steps:
  • Scope the problem: Don't make assumptions; Ask questions; Understand the constraints and use cases.
  • Sketch up an abstract design that illustrates the basic components of the system and the relationships between them.
  • Think about the bottlenecks these components face when the system scales.
  • Address these bottlenecks by using the fundamentals principles of scalable system design.
About design:
  • Everything is a tradeoff
  • there is no one optimal system design.
  • staying up to date


At the interview

First of all, follow the System Design Process. You already know how to apply it, so we'll be brief. Don't skip steps, don't make assumptions, start broad and go deep when asked.

Second, keep in mind that system design questions serve as an idea exchange platform. Be prepared for discussions about tradeoffs, about pros and cons. Be prepared to give alternatives, to ask questions, to identify and solve bottlenecks, to go broad or deep depending on your interviewer's preferences.

Don't get defensive: whenever your interviewer challenges your architectural choices, acknowledge that rarely an idea is perfect, and outline the advantages and disadvantages of your choice. Be open to new constraints to pop up during the discussion and to adjust your architecture on the fly.


1. Use cases
2. Constraints (Math)
3. Abstract design
    • Outline all the important components that your architecture will need.
    • Sketch your main components and the connections between them. 
    • If you do this, very quickly you will be able to get feedback if you are moving in the right direction.
    • Of course, you must be able to justify the high-level design that you just drew.
    • Make sure you sketch the important components and the connections between them.
    • Justify your ideas in front of the interviewer and try to address every constraint and use case.
4. Understanding bottlenecks
    • Needs to be scalable, in order for you to be able to improve it using some standard tools and techniques.
    • It may be the case that the interviewer wants to direct the discussion in one particular direction.
    • Then, maybe you won't need to address all the bottlenecks but rather talk in more depth about one particular area.
    • In any case, you need to be able to identify the weak spots in a system and be able to resolve them.
    • Remember, usually each solution is a trade-off of some kind. Changing something will worsen something else.
    • However, the important thing is to be able to talk about these trade-offs, and to measure their impact on the system given the constraints and use cases defined.
5. Scaling your abstract design


Scale:
  • Vertical scaling
  • Horizontal scaling
  • Caching / Sticky Session
  • Load balancing
  • Database replication
  • Database partitioning

Divide and Conquer - The Scalability Technique

  • This is the scalability technique. Everything is about partitioning out work. Deciding how to execute it. Applies to many things, from web tier, you have a lot of web servers that are more or less identically and independently and you grow them horizontally. That’s divide and conquer.
  • This is the crux of database sharding. How do you partitions things out and communicate between the parts that you’ve subdivided. These are things you want to figure out early on because they influence how you grow.
  • Simple and loose connections are really valuable.
  • The dynamic nature of Python is a win here. No matter how bad your API is you can stub or modify or decorate your way out of a lot of problems.

Avoid Thundering herd problem

Same as design in RAFT, use random!

May 8, 2018

[distributed system build up][book reading notes] Designing distributed system

Sidekick pattern


Ambassadors


Adapter


Replicated Load-Balanced Services

  • Stateless Services

  • Session Tracked Services
    • Session tracking is accomplished via a consistent hashing function

  • Application-Layer Replicated Services
  • Introducing a Caching Layer
    • A cache exists between your stateless application and the end-user request. 

    • Deploy using the sidecar pattern

Using https://varnish-cache.org/ for HTTP cacheing



Sharded Services


  • Sharded Caching
Many sharding functions use consistent hashing functions. 
Consistent hashing functions are special hash functions that are guaranteed to only remap # keys / # shards, when being resized to # shards.

For example, if we use a consistent hashing function for our sharded cache, moving from 10 to 11 shards will only result in remapping < 10% (K / 11) keys.
This is dramatically better than losing the entire sharded service.

The performance of your cache is defined in terms of its hit rate. 

The hit rate is the percentage of the time that your cache contains the data for a user request.

Ultimately, the hit rate determines the overall capacity of your distributed system and affects the overall capacity and performance of your system.

Sharding Functions
Shard = ShardingFunction(Req)
or, for programming languages,
Shard = hash(Req) % 10

Commonly, the sharding function is defined using a hashing function and the modulo(%) operator.

Hashing functions are functions that transform an arbitrary object into an integer hash.
The hash function has two important characteristics for our sharding:
  • Determinism
    • The output should always be the same for a unique input.
  • Uniformity
    • The distribution of outputs across the output space should be equal.
Selecting a Key
A better sharding function would be shard(request.path). 

When we use request.path as the shard key, then we map both requests to the same shard, and thus the response to one request can be served out of the cache to service the other.


Hot Sharding Systems
Ideally the load on a sharded cache will be perfectly even, but in many cases this isn't true and “hot shards” appear because organic load patterns drive more traffic to one particular shard.

As an example of this, consider a sharded cache for a user's photos; when a particular photo goes viral and suddenly receives a disproportionate amount of traffic, the cache shard containing that photo will become “hot.”

When this happens, with a replicated, sharded cache, you can scale the cache shard to respond to the increased load.

Indeed, if you set up auto scaling for each cache shard, you can dynamically grow and shrink each replicated shard as the organic traffic to your service shifts around.

An illustration of this process is shown in Figure 6-3. Initially the sharded service receives equal traffic to all three shards.

Then the traffic shifts so that Shard A is
receiving four times as much traffic as Shard B and Shard C.

The hot sharding system moves Shard B to the same machine as Shard C, and replicates Shard A to a second machine.

Traffic is now, once again, equally shared between replicas.



Scatter/Gather

Uses replication for scalability in terms of time.

Like replicated and sharded systems, the scatter/gather pattern is a tree pattern with a root that distributes requests and leaves that process those requests.

However, in contrast to replicated and sharded systems, with scatter/gather requests are simultaneously farmed out to all of the replicas in the system.

Each replica does a small amount of processing and then returns a fraction of the result to the root.

The root server then combines the various partial results together to form a single complete response to the request and then sends this request back out to the client.


example:


scatter/gather systems lead us to some conclusions:
  • Increased parallelism doesn't always speed things up because of overhead on each node.
  • Increased parallelism doesn't always speed things up because of the straggler problem. (one process is slow)
  • The performance of the 99th percentile is more important than in other systems because each user request actually becomes numerous requests to the service.
The same straggler problem applies to availability.

If you issue a request to 100 leaf nodes, and the probability that any leaf node failing is 1 in 100, you are again practically guaranteed to fail every single user request.

Below:
Built this way, each leaf request from the root is actually load balanced across all healthy replicas of the shard. 

This means that if there are any failures, they won't result in a user visible outage for your system.

Likewise, you can safely perform an upgrade under load, since each replicated shard can be upgraded one replica at a time.

Indeed, you can perform the upgrade across multiple shards simultaneously, depending on how quickly you want to perform the upgrade.




Functions and Event-Driven Processing

Function-as-a-service (FaaS)

When FaaS Makes Sense:
  • Functions are stateless and thus any system you build on top of functions is inherently more modular and decoupled than a similar system built into a single binary. 
  • Each function is entirely independent.
  • The only communication is across the network, 
  • And each function instance cannot have local memory, requiring all states to be stored in a storage service.
  • Additionally, the request-based and serverless nature of functions means that certain problems are quite difficult to detect.
  • FaaS is inherently an event-based application model. Functions are executed in response to discrete events that occur and trigger the execution of the functions.

The Decorator Pattern: Request or Response Transformation



Kubeless is deployed on top of the Kubernetes
container orchestration service. Assuming that you have provisioned a Kubernetes cluster, you can install Kubeless from its releases page. 
Once you have the kubeless binary installed, you can
install it into your cluster with the following cmd: 
kubeless install

To can see deployed functions cmd:
kubectl get functions.

Handling Events

Ownership Election

Work Queue Systems

In the containerized work queue, there are two interfaces:
  • the source container interface(like event sourcing), which provides a stream(也就是log, unbounded, immutable logs) of work items that need processing,
  • and the worker container interface, which knows how to actually process a work item.
URLs:
  • GET http://localhost/api/v1/items
  • GET http://localhost/api/v1/items/<item-name>









Jan 23, 2018

[C++17] C++17 - The Complete Guide [note]

C++17 - The Complete Guide


structured bindings do not decay. 

i.e
struct S {
const char x[6];
const char y[3];
};

S s1{};
auto [a, b] = s1; // a and b get the exact member types
the type of a still is const char[6].

This is different from initializing a new object with auto, where types decay:
auto a2 = a; // a2 gets decayed type of a
std::decay
MyStruct ms = { 42, "Jim" };
auto&& [v,n] = std::move(ms);
此時 anonymous entity r-value reference to ms.
v,n r-value reference to anonymous entity.

We can also capture as a forwarding reference using auto&&.
That is, auto&& will resolve to auto& for lvalue references, and auto&& for rvalue references.
MyStruct ms = { 42, "Jim" };
auto [v,n] = std::move(ms);  // new entity with moved-from values from ms

此時 anonymous entity 由 move constructor construct from ms.
也就是 ms 的內容已經 invalidate.

Note that there is only limited usage of inheritance possible.
All non-static data members must be members of the same class definition.
struct B {
int a = 1;
int b = 2;
};

struct D1 : B {
};

auto [x, y] = D1{}; // OK

struct D2 : B {
int c = 3;
};
auto [i, j, k] = D2{}; // Compile-Time ERROR
std::pair
std::tuple
std::array

array:
std::array<int,4> getArray();
auto [i,j,k,l] = getArray();

tuple:
std::tuple<char,float,std::string> getTuple();
auto [a,b,c] = getTuple();
pair:
std::map<std::string, int> coll;
...
auto [pos,ok] = coll.insert({"new",42});
if (!ok) {
// if insert failed, handle error using iterator pos:
...
}
or
if (auto [pos,ok] = coll.insert({"new",42}); !ok) {
// if insert failed, handle error using iterator pos:
const auto& [key,val] = *pos;
std::cout << "already there: " << key << ’\n’;
}


Aggregate Extensions

struct Data {
std::string name;
double value;
};
Data x{"test1", 6.778};


struct MoreData : Data {
bool done;
};
MoreData y{{"test1", 6.778}, false};
struct Data {
const char* name;
double value;
};
struct PData : Data {
bool critical;
void print() const {
std::cout << ’[’ << name << ’,’ << value << "]\n";
}
};
PData y{{"test1", 6.778}, false};
y.print();

PData a{}; // zero-initialize all elements
PData b{{"msg"}}; // same as {{"msg",0.0},false}
PData c{{}, true}; // same as {{nullptr,0.0},true}
PData d; // values of fundamental types are unspecified
template<typename T>
struct D : std::string, std::complex<T>
{
std::string data;
};

D<float> s{{"hello"}, {4.5,6.7}, "world"}; // OK since C++17
std::cout << s.data; // outputs: ”world”
std::cout << static_cast<std::string>(s); // outputs: ”hello”
std::cout << static_cast<std::complex<float>>(s); // outputs: (4.5,6.7)

Definition of Aggregates

  • either an array
  • or a class type (class, struct, or union) with:
    • no user-declared or explicit constructor
    • no constructor inherited by a using declaration
    • no private or protected non-static data members
    • no virtual functions
    • no virtual, private, or protected base classes
To be able to use an aggregate it is also required that
  • no private or 
  • protected base class members
  • or constructors
 are used during initialization.

is_aggregate<>
struct Derived;

struct Base {
friend struct Derived;
private:
Base() {
}
};

struct Derived : Base {
};

int main()
{
Derived d1{}; // ERROR since C++17
Derived d2; // still OK (but might not initialize)
}

Before C++17, Derived was not an aggregate. Thus,
Derived d1{};
was calling the implicitly defined default constructor of Derived, which by default called the default constructor of the base class Base.

Although the default constructor of the base class is private, it was valid to be called as via the default constructor of the derived class, because the derived class was defined to be a friend class.

Since C++17, Derived in this example is an aggregate, not having an implicit default constructor
at all (the constructor is not inherited by a using declaration).

So the initialization is an aggregate initialization, for which it is not allowed to call private constructors of bases classes. Whether the base class is a friend doesn't matter.

RVO

http://en.cppreference.com/w/cpp/language/copy_elision

Copy ellision works even copy/move constructor is deleted.
class MyClass
{
public:
...
// no copy/move constructor defined:
MyClass(const MyClass&) = delete;
MyClass(MyClass&&) = delete;
...
};

// Works even no copy constructor
void foo(MyClass param) {}

// Works even no copy constructor
MyClass bar() {
return MyClass();
}

NRVO

Copy/move constructor still needs to be around.

MyClass foo()
{
MyClass obj;
...
return obj; // still requires copy/move support
}
-
MyClass bar(MyClass obj) // copy elision for passed temporaries
{
...
return obj; // still requires copy/move support
}

So, how this benefits us?

Any type that is _not_ copyable can be used in a factory function
which return the type instance, which is not going to be copied!
#include <utility>
template <typename T, typename... Args>
T create(Args&&... args)
{
...
return T{std::forward<Args>(args)...};
}


int i = create<int>(42);
std::unique_ptr<int> up = create<std::unique_ptr<int>>(new int{42});
std::atomic<int> ai = create<std::atomic<int>>(42);

Also, any type with it's move constructor deleted, as in factory function,
we still can pass a r-value instance back.
class CopyOnly {
public:
CopyOnly() {
}
CopyOnly(int) {
}
CopyOnly(const CopyOnly&) = default;
CopyOnly(CopyOnly&&) = delete; // explicitly deleted
};

CopyOnly ret() {
return CopyOnly{}; // OK since C++17
}
CopyOnly x = 42; // OK since C++17


Value Categories


It's worth emphasizing that strictly speaking glvalues, prvalues, and xvalues are terms for expressions and not for values.

A variable itself is not an lvalue;
only an expression denoting the variable is an lvalue:
int x = 3; // x here is a variable, not an lvalue
int y = x; // x here is an lvalue

In the first statement 3 is a prvalue initializing the variable (not the lvalue) x.
In the second statement x is an lvalue (its evaluation designates an object containing the value 3). The lvalue x is converted to a prvalue, which is what initializes the variable y.

The key approach to explain value categories now is that in general we have two kinds of expressions:
  • glvalues: expressions for locations of objects or functions
  • prvalues: expressions for initializations
An xvalue is then considered a special location, representing an object whose resources can be reused (usually because it is near the end of its lifetime).

C++17 then introduces a new term, called materialization (of a temporary) for the moment a prvalue becomes a temporary object.
Thus, a temporary materialization conversion is a prvalue-to-xvalue conversion.

Lambda
It's constexpr iff it doesn't capture.

auto squared = [](auto val) {
 // implicitly constexpr since C++17
return val*val;
};
std::array<int,squared(5)> a; // OK since C++17 => std::array<int,25>
auto squared2 = [](auto val) {
 // implicitly constexpr since C++17
static int calls = 0;
 // OK, but disables lambda for constexpr contexts
...
return val*val;
};
std::array<int,squared2(5)> a;
 // ERROR: static variable in compile-time context
std::cout << squared2(5) << ’\n’; // OK

如何測試lambda expression 為constexpr? 

將lambda expression 冠上constexpr,如果compile出錯,則非constexpr.

Evaluation Order


To fix all this unexpected behavior, for some operators the evaluation guarantees were refined so
that they now specify a guaranteed evaluation order:
For:
e1 [ e2 ]
e1 . e2
e1 .* e2
e1 ->* e2
e1 << e2
e1 >> e2
e1 is guaranteed to get evaluated before e2 now, so that the evaluation order is left to right.


However, note that the evaluation order of different arguments of the same function call is still undefined. 
That is, in
e1.f(a1,a2,a3)
e1 is guaranteed to get evaluated before a1, a2, and a3 now. However, the evaluation order of a1,
a2, and a3 is still undefined.
In all assignment operators:
e2 = e1
e2 += e1
e2 *= e1
...
the right-hand side e1 is guaranteed to get evaluated before the left-hand side e2 now.
In new expressions like: (重要!)
new Type(e)

the allocation is now guaranteed to be performed before the evaluation e, and the initialization of the new value is guaranteed to happen before any usage of the allocated and initialized value.


Enum Initialization from Integral Values

For enumerations with a fixed underlying type, since C++17 you can use an integral value of that type for direct list initialization. 

This applies to unscoped enumerations with a specified type and all scoped enumerations, because they always have an underlying default type:
unscoped enum with underlying type
enum MyInt : char { };
MyInt i1{42}; // OK since C++17 (ERROR before C++17)
MyInt i2 = 42; // still ERROR
MyInt i3(42); // still ERROR
MyInt i4 = {42}; // still ERROR
scoped enum with default underlying type
enum class Salutation { mr, mrs };
Salutation s1{0}; // OK since C++17 (ERROR before C++17)
Salutation s2 = 0; // still ERROR
Salutation s3(0); // still ERROR
Salutation s4 = {0}; // still ERROR
The same applies if Salutation has a specified underlying type:

scoped enum with specified underlying type
enum class Salutation : char { mr, mrs };
Salutation s1{0}; // OK since C++17 (ERROR before C++17)
Salutation s2 = 0; // still ERROR
Salutation s3(0); // still ERROR
Salutation s4 = {0}; // still ERROR

For unscoped enumerations (enum without class) having no specified underlying type,
you still can't use list initialization for numeric values
enum Flag { bit1=1, bit2=2, bit3=4 };
Flag f1{0}; // still ERROR
Note also that list initialization still doesn't allow narrowing, so you can't pass a floating-point value.
enum MyInt : char { };
MyInt i5{42.2}; // still ERROR


Fixed Direct List Initialization with auto

int x{42}; // initializes an int
int y{1,2,3}; // ERROR
auto a{42}; // initializes an int now
auto b{1,2,3}; // ERROR now

auto c = {42}; // still initializes a std::initializer_list<int>
auto d = {1,2,3}; // still OK: initializes a std::initializer_list<int>

auto a{42}; // initializes an int now
auto c = {42}; // still initializes a std::initializer_list<int>


Since C++17 exception handling specifications became part of the type of a function

void f1();
void f2() noexcept;// different type

void (*fp)() noexcept; // pointer to function that doesn't throw
fp = f2; // OK
fp = f1; // ERROR since C++17

void (*fp2)(); // pointer to function that might throw
fp2 = f2; // OK
fp2 = f1; // OK

It is not allowed to overload a function name for the same signature with a different exception specification (as it is not allowed to overload functions with different return types only):

void f3();
void f3() noexcept; // ERROR
也就是對於template T而言以上兩個function為不同type.
在template programming中,注意noexcept也為type的一部分!

__has_include

#if __has_include(<filesystem>)
# include <filesystem>
# define HAS_FILESYSTEM 1
#elif __has_include(<experimental/filesystem>)
# include <experimental/filesystem>
# define HAS_FILESYSTEM 1
# define FILESYSTEM_IS_EXPERIMENTAL 1
#elif __has_include("filesystem.hpp")
# include "filesystem.hpp"
# define HAS_FILESYSTEM 1
# define FILESYSTEM_IS_EXPERIMENTAL 1
#else
# define HAS_FILESYSTEM 0
#endif


Deduction Guides

We can define specific deduction guides to provide additional or fix existing class template argument deductions. 

e.g:
template<typename T>
struct C {
C(const T&) {
}
...
};

C x{"hello"}; // T deduced as const char[6]

// now we do
template<typename T> C(T) -> C<T>;

C x{"hello"}; // T deduced as const char*
i.e 重要!
A corresponding deduction guide sounds very reasonable for any class template having a constructor taking an object of its template parameter by reference.

Non-Template Deduction Guides

template<typename T>
struct S {
T val;
};

S(const char*) -> S<std::string>; // map S<> for string literals to S<std::string>
Note that aggregates need list initialization
(the deduction works, but the initialization is not allowed):
S s4 = "hello"; // ERROR (can’t initialize aggregates that way)
Deduction guides compete with the constructors of a class.
Class template argument deduction uses the constructor/guide that has the highest priority according to overload resolution.

If a constructor and a deduction guide match equally well, the deduction guide is preferred.


Explicit Deduction Guides

template<typename T>
struct S {
T val;
};
explicit S(const char*) -> S<std::string>;

S s1 = {"hello"}; // ERROR (deduction guide ignored and otherwise invalid)

S s2{"hello"}; // OK, same as: S<std::string> s1{"hello"};
S s3 = S{"hello"}; // OK
S s4 = {S{"hello"}}; // OK
another e.g
template<typename T>
struct Ptr
{
Ptr(T) { std::cout << "Ptr(T)\n"; }
template<typename U>
Ptr(U) { std::cout << "Ptr(U)\n"; }
};
template<typename T>
explicit Ptr(T) -> Ptr<T*>;

Ptr p1{42}; // deduces Ptr<int*> due to deduction guide
Ptr p2 = 42;    // deduces Ptr<int> due to constructor
int i = 42;
Ptr p3{&i};     // deduces Ptr<int**> due to deduction guide
Ptr p4 = &i;    // deduces Ptr<int*> due to constructor


Deduction Guides for Aggregates

template<typename T>
struct A {
T val;
};

A i1{42}; // ERROR
A s1("hi"); // ERROR
A s2{"hi"}; // ERROR
A s3 = "hi"; // ERROR
A s4 = {"hi"}; // ERROR

A(const char*) -> A<std::string>;

A s2{"hi"}; // OK
A s4 = {"hi"}; // OK

Standard Deduction Guides Deduction from Iterators

namespace std {
template<typename Iterator>
vector(Iterator, Iterator)
-> vector<typename iterator_traits<Iterator>::value_type>;
}

std::set<float> s;
std::vector(s.begin(), s.end()); // OK, deduces std::vector<float>


std::array<> Deduction

std::array a{42,45,77}; // OK, deduces std::array<int,3>
namespace std {
template<typename T, typename... U>
array(T, U...)
-> array<enable_if_t<(is_same_v<T,U> && ...), T>,
(1 + sizeof...(U))>;
}


if constexpr
注意,任何type T dependable expression can be failed which is OK.
Expression without T dependable will not compile if the expression is itself invalid.
No Short-Circuit Compile-Time Conditions.


Fold Expressions

template<typename... T>
auto foldSum (T... args) {
return (... + args); // ((arg1 + arg2) + arg3) ...
}

Motivation for Fold Expressions:

Since before, we can't extract Arg... but only to implement function with recursive calls and retract argument one by one.
template<typename T>
const T& spaceBefore(const T& arg) {
std::cout << ’ ’;
return arg;
}

template <typename First, typename... Args>
void print (const First& firstarg, const Args&... args) {
std::cout << firstarg;
(std::cout << ... << spaceBefore(args)) << ’\n’;
}

// std::cout << spaceBefore(arg1) << spaceBefore(arg2) << ...

Supported Operators: 

all binary operators for fold expressions except .
->
[]
Fold expression can also be used for the comma operator, combining multiple expressions into one statement.
// template for variadic number of base classes
template<typename... Bases>
class MultiBase : private Bases...
{
public:
void print() {
// call print() of all base classes:
(... , Bases::print());
}
};


Dealing with Strings as Template Parameters 

Non-type template parameters can be only
  • constant integral values (including enumerations),
  • pointers to objects/functions/members,
  • lvalue references to objects
  • or functions, or std::nullptr_t (the type of nullptr)
For pointers, linkage is required, which means that you can't pass string literals directly.

However, since C++17, you can have pointers with internal linkage.

For example:
template<const char* str>
class Message {
...
};

extern const char hello[] = "Hello World!";     // external linkage
const char hello11[] = "Hello World!";              // internal linkage

void foo()
{
Message<hello>  msg;              // OK (all C++ versions)
Message<hello11> msg11;     // OK since C++11

static const char hello17[] = "Hello World!";       // no linkage

Message<hello17> msg17;         // OK since C++17
}

--
template<int* p> struct A {
};

int num;
A<&num> a;  // OK since C++11


--
int num;

constexpr int* pNum() {
return & num;
}

A<pNum()> b;  // ERROR before C++17, now OK


Nov 25, 2017

[C++][Book read] C++ concurrency in Action, 2nd edition

std::thread::native_handle
std::thread::hardware_concurrency()

Aware of thread constructor:
It's passing argument to callable function as rvalue through std::decay_t<T>.
Thus, if callable function is taking an l-value reference, compile fails.

std::thread::id offer the complete set of comparison operators,
which provide a total ordering for all distinct values.

The Standard Library provides std::hash<std::thread::id> so that values of
type std::thread::id can be used as keys in the new unordered associative containers.

FP like functions:



Before calling thread.join(), things have to be considered all code path with:
  • Will the callable function throw?
  • If the caller thread throws, what happen if thread.join() not called.
  • Using RAII
For thread's callable function's arguments:
by default the arguments are copied into internal storage,
where they can be accessed by the newly created thread of execution,
and then passed to the callable object or function as rvalues as if they were temporaries.
Thus, use
std::ref

reference boost::bind:
http://vsdmars.blogspot.com/2013/06/cboost-lambda-note.html
mem_fn

Sharing data between threads:
If all shared data is read-only, there's no problem, because
the data read by one thread is unaffected by whether or not another thread is reading the
same data.
i.e
a const member function implies thread safe.


Sharing data between threads:
mutex:

std::mutex some_mutex;
std::lock_guard<std::mutex> guard(some_mutex);
std::lock(lhs.m,rhs.m);
# instance of std::adopt_lock_t http://en.cppreference.com/w/cpp/thread/lock_tag_t
std::lock_guard<std::mutex> lock_a(lhs.m,std::adopt_lock);
std::lock_guard<std::mutex> lock_b(rhs.m,std::adopt_lock);

std::lock
std::scoped_lock RAII style.

Race conditions:
Avoiding problematic race conditions:
  1. Wrap data structure with a protection mechanism, to ensure that only the thread actually performing a modification can see the intermediate states where the invariants are broken.
  2.  Modify the design of your data structure and its invariants so that modifications are done as a series of indivisible changes, each of which preserves the invariants. This is generally referred to as lock-free programming.
  3. Handle the updates to the data structure as a transaction, just as updates to a database are done within a transaction. The required series of data modifications and reads is stored in a transaction log and then committed in a single step. If the commit can’t proceed because the data structure has been modified by another thread, the transaction is restarted. This is termed software transactional memory (STM), and it’s an active research area at the time of writing.
Aware of constructo might throw, which makes the container's data loss.
Thus solution:
  • PASS IN A REFERENCE
  • REQUIRE A NO-THROW COPY CONSTRUCTOR OR MOVE CONSTRUCTOR
  • RETURN A POINTER TO THE POPPED ITEM
  • PROVIDE BOTH OPTION 1 AND EITHER OPTION 2 OR 3

The class unique_lock is a general-purpose mutex ownership wrapper allowing deferred locking,
time-constrained attempts at locking, recursive locking, transfer of lock ownership,
and use with condition variables.

RWLock:
The class shared_lock is a general-purpose shared mutex ownership wrapper allowing deferred locking, timed locking and transfer of lock ownership. Locking a shared_lock locks the associated shared mutex in shared mode (to lock it in exclusive mode, std::unique_lock can be used)

std::unique_lock<std::mutex> lock_a(lhs.m,std::defer_lock); // http://en.cppreference.com/w/cpp/thread/unique_lock
std::unique_lock<std::mutex> lock_b(rhs.m,std::defer_lock);
std::lock(lock_a,lock_b); // http://en.cppreference.com/w/cpp/thread/lock


mutex:

has two levels of access:
  • shared - several threads can share ownership of the same mutex.
  • exclusive - only one thread can own the mutex.
Shared mutexes are usually used in situations when multiple readers can access the same resource at the same time without causing data races, but only one writer can do so.


Most of the time, if you think you want a recursive mutex, you probably need to change
your design instead. A common use of recursive mutexes is where a class is designed to be
accessible from multiple threads concurrently, so it has a mutex protecting the member data.



Ch. 4

Synchronizing concurrent operations:

header
<condition_variable>

std::condition_variable is preferred then std::condition_variable_any.

Pattern:

Producer:

std::lock_guard

modify data.
unlock mutex.
std::condition_variable notify_one 

Waiter:

std::unique_lock
std::condition_variable wait 
modify data.
unlock mutex.

header
<future>

std::async
Just as with std::thread, if the arguments are rvalues,
the copies are created by moving the originals.
This allows the use of move-only types as both the function
object and the arguments.

#include <string>
#include <future>

struct X
{
void foo(int,std::string const&);
std::string bar(std::string const&);
};

X x;

auto f1=std::async(&X::foo,&x,42,"hello");  // Calls p->foo(42,"hello") where p is &x
auto f2=std::async(&X::bar,x,"goodbye");    // Calls tmpx.bar("goodbye") where tmpx is a copy of x

struct Y
{
double operator()(double);
};
Y y;

auto f3=std::async(Y(),3.141);  // Calls tmpy(3.141) where tmpy is move-constructed from Y()
auto f4=std::async(std::ref(y),2.718);  // Calls y(2.718)

X baz(X&);

std::async(baz,std::ref(x));    // Calls baz(x)

class move_only
{
public:
move_only();
move_only(move_only&&)
move_only(move_only const&) = delete;
move_only& operator=(move_only&&);
move_only& operator=(move_only const&) = delete;
void operator()();
};

auto f5=std::async(move_only());    // Calls tmp() where tmp is constructed from std::move(move_only())

std::packaged_task

The std::packaged_task object is thus a callable object, and it can be wrapped in a
std::function object, passed to a std::thread as the thread function, passed to another
function that requires a callable object, or even invoked directly.

std::promise

some_promise.set_exception(std::make_exception_ptr(std::logic_error("foo ")));

Another way to store an exception in a future is to destroy the std::promise or
std::packaged_task associated with the future without calling either of the set functions on
the promise or invoking the packaged task.
In either case, the destructor of the std::promise or std::packaged_task will store a
std::future_error exception with an error code of std::future_errc::broken_promise
 in the associated state if the future isn’t already ready;

std::future


// get shared_future
std::promise< std::map< SomeIndexType, SomeDataType, SomeComparator,
SomeAllocator>::iterator> p;
auto sf=p.get_future().share();

C++ time class:

namespapce
std::literals::chrono_literals 
contains literals and chrono_literals
std::ratio has predefined type.
using namespace std::literals::chrono_literals
using namespace std::literals
using namespace std::chrono_literals
Fixed width integer types

Duration literals
user defined literals from cppref and c++11 faq
 
There are four kinds of literals that can be suffixed to make a user-defined literal:
  • integer literal: accepted by a literal operator taking a single unsigned long long or const char* argument.
  • floating-point literal: accepted by a literal operator taking a single long double or const char* argument.
  • string literal: accepted by a literal operator taking a pair of (const char*, size_t) arguments.
  • character literal: accepted by a literal operator taking a single char argument.

using namespace std::chrono_literals;
auto one_day=24h;
auto half_an_hour=30min;
auto max_time_between_messages=30ms;

Explicit conversions can be done with std::chrono::duration_cast<>
std::chrono::milliseconds ms(54802);
std::chrono::seconds s;
std::chrono::duration_cast<std::chrono::seconds>(ms);

Time points

std::chrono::time_point<>


header:

<experimental/future> 

std::experimental::when_all
std::experimental::when_any

std::experimental::latch
std::experimental::barrier
more basic, and potentially therefore has lower overhead

std::experimental::flex_barrier
more flexible, but potentially has more overhead.