[java] avoid these smells

For high performance/innovative projects, use Go/Rust.
For job
here are some bad smells of Java which should be avoid:

1.
https://commons.apache.org/proper/commons-collections/javadocs/api-3.2.2/org/apache/commons/collections/CollectionUtils.html
CollectionUtils.isEmpty/isNotEmpty for checking container's emptiness and nullness

2.
This is simple idea in other languages.
For dynamic sized containers (C++'s vector), pre-assign size before using it
to prevent copy-and-extend

3.
Use StringBuilder since everything in Java is allocated on heap..

4.
Like C++ but more uglier(Java has this check at runtime, not compile time), Java has the concept of type of iterator.
If the passing in List<> is RandomAccess implemented, it can be traverse
with random location(e.g ArrayList/Stack/Vector,  C++'s Array/Vector)
https://docs.oracle.com/javase/8/docs/api/java/util/RandomAccess.html

if (list instanceof RandomAccess) ...

5.
Don't use fancy but awful performance anonymous type trick:
List<String> l = new ArrayList<>{
    add(1);
    add(3);
    };

6.
Use try-with-resource
https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html

Type implements:
1. java.io.Closeable
2. java.lang.AutoCloseable
Should use try-with-resource.

try-with resource will not suppress try block's exception which only throws
'finally's exception, this is unlike the use of try/finally

For retrieving suppress exception use:
Throwable.getSuppressed

7.
Java is full of sxxx, no, full of patterns
For Util class(do not use 'util' in Go, it doesn't make sense at all)
hide its constructor.

8.
Use:
Object.isNull everywhere

9.
Use:
String.valueOf

10.
Use:
@Deprecated

11.
Do not compare with different type with <>= operator
It's common concept in strong type languages.

12.
Return empty container instead of null.
Null itself is not part of the type of the container.
Use:
Collections.emptyXXX

13.
Again, beware function argument's nullness.
John Lakos' defensive programming is a good thing for Java engineers.
https://vsdmars.blogspot.com/2015/01/defensive-programming.html

14.
String.split() takes regex, beware to escape if necessary
~RTFM~

15.
Favor unchecked RuntimeException.
Checked exception is by design a flaw which Anders Hejlsberg has mentioned this almost 20 years ago:
https://www.artima.com/intv/handcuffs.html
C++ has dumped runtime exception and favor compile time check noexcept
some reference here:
https://stackoverflow.com/a/2190177
https://phauer.com/2015/checked-exceptions-are-evil/

16.
Unit test rule
A:Automatic(自动化)
I:Independent(独立性)
R:Repeatable(可重复)

17.
Unit test principle
B:Border，边界值测试，包括循环边界、特殊取值、特殊时间点、数据顺序等。
C:Correct，正确的输入，并得到预期的结果。
D:Design，与设计文档相结合，来编写单元测试。
E:Error，强制错误信息输入(如:非法数据、异常流程、业务允许外等)，并得到预期的结果。

[java] lambda translation

Reference:
https://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html

Languages like C++, Java, which lately added support to lambda expression, internally compiler front-end generates mapping code for backend to parse and consume.

C++ generates lambda expression into type, with name mangling, provides most powerful way to capture/reference/move variables outside the lambda expression scope.

Java, OTHO, generates lambda expression to method, duh~


Stateless:
Like C++, Java generates lambda expression into static member function.
C++: http://vsdmars.blogspot.com/2016/07/c-non-capturing-c-lambdas-can-be.html

class A {
    public void foo() {
        List<string> list = ...
        list.forEach( s -> { System.out.println(s); } );
    }
}

to:

class A {
    public void foo() {
        List<string> list = ...
        list.forEach( [lambda for lambda$1 as Block] );
    }

    static void lambda$1(String s) {
        System.out.println(s);
    }
}

Capturing immutable values:
Unlike Go, which is smarter by replacing captured variable by value iff it's not referenced in later scope, Java does it by generate a static function with parameters copy by value.

class B {
    public void foo() {
        List<person> list = ...
        final int bottom = ..., top = ...;
        list.removeIf( p -> (p.size >= bottom && p.size <= top) );
    }
}

to:

class B {
    public void foo() {
        List<person> list = ...
        final int bottom = ..., top = ...;
        list.removeIf( [ lambda for lambda$1 as Predicate capturing (bottom, top) ]);
    }

    static boolean lambda$1(int bottom, int top, Person p) {
        return (p.size >= bottom && p.size <= top;
    }
}

The Lambda Metafactory:
Lambda capture will be implemented by an invokedynamic call site, whose static parameters describe the characteristics of the lambda body and lambda descriptor, and whose dynamic parameters (if any) are the captured values.

When invoked, this call site returns a lambda object for the corresponding lambda body and descriptor, bound to the captured values.

The bootstrap method for this callsite is a specified platform method called the lambda metafactory. (We can have a single metafactory for all lambda forms, or have specialized versions for common situations.)

The VM will call the metafactory only once per capture site; thereafter it will link the call site and get out of the way.

Call sites are linked lazily, so factory sites that are never invoked are never linked.

The static argument list for the basic metafactory looks like: 

metaFactory(MethodHandles.Lookup caller, // provided by VM
            String invokedName,          // provided by VM
            MethodType invokedType,      // provided by VM
            MethodHandle descriptor,     // lambda descriptor
            MethodHandle impl)           // lambda body

What about capturing type instance's value?

list.filter(e -> e.getSize() < minSize )

to:

list.forEach(INDY((MH(metaFactory), MH(invokeVirtual Predicate.apply),
                   MH(invokeVirtual B.lambda$1))( this ))));

private boolean lambda$1(Element e) {
    return e.getSize() < minSize;
}

EOF

As it shows, Java doesn't give you much control over instance's capture...

[java] type instance initialization constructor does not do inline.

Reference:
https://pangin.pro/posts/computation-in-static-initializer
https://cl4es.github.io/2019/02/21/Cljinit-Woes.html


HotSpot does not inline methods of uninitialized classes.
source code:
https://hg.openjdk.java.net/jdk-updates/jdk11u/file/cd1c042181e9/src/hotspot/share/opto/bytecodeInfo.cpp#l455

Access to a static field from a static method of uninitialized class may be an overwhelming obstacle for HotSpot compiler.

Work around:
Just don't do heavy computation in an uninitialized class directly. :-\
(From compiled language point of view, it's quite absurd and duh~duh~duh~)

If putting the computation logic in a helper class with no static initializer, it won’t suffer from performance penalty.

e.g

public class StaticExample {
    static final long[] TABLE = Helper.prepareTable();

    private static class Helper {

        static long[] prepareTable() {
            long[] table = new long[100_000_000];
            for (int i = 1; i < table.length; i++) {
                table[i] = nextValue(table[i - 1]);
            }
            return table;
        }

        static long nextValue(long seed) {
            return seed * 0x123456789L + 11;
        }
    }
}

[java] memory - note

java, a good example of unnecessary complexity due to unthoughtful design.

Debian programming language game, Go vs. Java, look at the memory foot-print..
Go has a GC and due to it's language design, GC can be implemented much lighter-weight.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/go.html


Materials:
------------------------------
Java 8: From PermGen to Metaspace:
http://javaeesupportpatterns.blogspot.com/2013/02/java-8-from-permgen-to-metaspace.html


Java Performance Optimization:
https://dzone.com/refcardz/java-performance-optimization?chapter=1

flame-graph
http://javaeesupportpatterns.blogspot.com/2015/09/java-8-cpu-flame-graph.html

Java in Flames-netflix
https://medium.com/netflix-techblog/java-in-flames-e763b3d32166

ref
http://javaeesupportpatterns.blogspot.com/2015/07/jvm-buzzwords-java-developers-should.html


5 tips for proper Java Heap size:
http://javaeesupportpatterns.blogspot.com/2012/07/5-tips-for-proper-java-heap-size.html

Java Heap Dump:
http://javaeesupportpatterns.blogspot.com/2012/11/java-heap-dump-are-you-up-to-task.html

[Go][Java]&[Clang] tail call optimization status in 2019

Go:
Not going to happen in Go2 as well.
https://github.com/golang/go/issues/22624

Java:
JVM doesn’t support Tail Call Optimization.
https://softwareengineering.stackexchange.com/a/272086
https://www.youtube.com/watch?v=2y5Pv4yN0b0&t=1h02m18s

Clang:
Tail call optimization, callee reusing the stack of the caller, is currently supported on x86/x86-64, PowerPC, and WebAssembly.
It is performed on x86/x86-64 and PowerPC if:

• Caller and callee have the calling convention fastcc, cc 10 (GHC calling convention) or cc 11 (HiPE calling convention). 
• The call is a tail call - in tail position (ret immediately follows call and ret uses value of call or is void). 
• Option -tailcallopt is enabled. 
• Platform-specific constraints are met. 

x86/x86-64 constraints: 

• No variable argument lists are used. 
• On x86-64 when generating GOT/PIC code only module-local calls (visibility = hidden or protected) are supported.

[Java] Java SE Arch

https://www.oracle.com/technetwork/java/javase/tech/index.html

[Go][Java][C++] Language Spec

C++
http://eel.is/c++draft/
https://timsong-cpp.github.io/cppwp/
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4659.pdf
http://www.open-std.org/jtc1/sc22/wg21/
https://isocpp.org/
https://stackoverflow.com/a/83763
https://en.cppreference.com

Java
https://docs.oracle.com/javase/specs/
https://docs.oracle.com/en/java/javase/12/

Go
https://golang.org/ref/spec

[C++] C++ Concurrency In Action, Second edition, recap [Ch.5]

Anthony Williams' C++ Concurrency In Action book hits second edition, hereby jotting down reading notes starts from Chapter 5, which draws the C++'s memory model and concurrent program design.
( The Art of Multiprocessor Programming , [multiprocessor programming] types of synchronization)

For the system languages I am currently using having the sequential-consistent memory model, which is quite straight forward to work with (Go, Java). While C++ gives us a bit more,
thus the understanding of MESI , store buffer,
memory/compiler barrier would be a plus for reading through the context.

[C++][Java] Kinds of variance in subtyping

Reference:
https://eli.thegreenplace.net/2018/covariance-and-contravariance-in-subtyping

I've to say I admire Eli Bendersky a lot. Not only he could make complicated stuff in plain English, but also broadly read.

Covariant is a well known concept among expert C++ engineers, which includes the type system as well as the memory model. (Object memory layout.)
Even in 2018, I would recommand anyone who are into programming languages to give a quick read on Stanley B, Lippman's Inside The C++ Object Model.

Liskov substitution principle

Given types S and T with the relation S <: T, variance is a way to describe the relation between the composite types:

• Covariant means the ordering of component types is preserved: Composite<S> <:Composite<T>.
  C++: works for function return type.
• Contravariant means the ordering is reversed: Composite<T> <: Composite<S>.
  C++: works for function parameter. i.e Argument type should be no deeper then function's parameter's signature type.
• Bivariant means both covariant and contravariant.
• Invariant means neither covariant nor contravariant.

Consider this code:

struct MammalClinic {
  virtual void Accept(Mammal* m);
};

struct CatClinic : public MammalClinic {
  virtual void Accept(Cat* c);
};

The CatClinic::Accept is an overload, not override, and this is exactly the keyword: 'override' is created for: to spot this kind of error.
The reality is that function overrides are not covariant for pointer types.
They are invariant.
In fact, the vast majority of typing rules in C++ are invariant;
std::vector<Cat> is not a subclass of std::vector<Mammal>, even though Cat <: Mammal.
There's a good reason for that.

Let's look into the example from Java:

class Main {
  public static void main(String[] args) {
    String strings[] = {"house", "daisy"};
    Object objects[] = strings; // covariant

    objects[1] = "cauliflower"; // works fine
    objects[0] = 5;             // throws exception at runtime.
  }
}

Assigning an integer fails because at run-time it's known that objects is actually an array of strings.
Thus, covariance together with mutability makes array types unsound.
Note, however, that this is not just a mistake - it's a deliberate historical decision made when Java didn't have generics and polymorphism was still desired; the same problem exists in C#.

Other languages have immutable containers, which can then be made covariant without jeopardizing the soundness of the type system.


Contravariant example in std::function:

#include <functional>

struct Vertebrate {};
struct Mammal : public Vertebrate {};
struct Cat : public Mammal {};

Cat* f1(Vertebrate* v) {
  return nullptr;
}

Vertebrate* f2(Vertebrate* v) {
  return nullptr;
}

Cat* f3(Cat* v) {
  return nullptr;
}

void User(std::function<Mammal*(Mammal*)> f) {
  // do stuff with 'f'
}

int main() {
  User(f1);       // works
  User(f2);       // return type covariance failed, since Vertebrate is base type of Mammal.
  User(f3);       // Argument type contravariance failed, since Cat is deeper then Mammal.

  return 0;
}

[Concurrency] [C++][Go] Wrap up - 2018

Modified

• Thie Core's cache line has the modified data.
• Data in memory won't be in other Core's cache line.

Exclusive

• Data aren't modified.
• Data in memory is the latest.
• If the Core has to evict the data inside the cache line,
  nothing need to be write back to memory.

Shared

• Data are shared between Cores' cache line.
• If this core has to modify the data, it needs to ask for data from other cores first.

Invalid

• The data in the cache line is null.

Operations:

Read

• read data from cache lines. Ask for other cores' for data.  

Read response

• Data for read request. Either from Memory or from Cores' cache.

Invalidate

• Invalidate the particular data inside Cores' cache lines.

Invalidate ack

• Response to invalidate request that the request is in the queue.      

Read invalidate

• Read + Invalidate.
• Will get read's data response and invalidate ack.

Writeback

• Write the data from cache line to memory.

Store buffer:

• Read invalidate request to other Cores' cache line for the to be modified data  is not necessary since the data response isn't needed because this Core is going to modify the data anyway.
• Thus, this core will write data to Store buffer first.
• Thus, Core will read data inside the Store Buffer with highest priority if the data exist in Store buffer and Cache line.
• However, there's an issue for Read invalidate request to other cores.
  Consider that data A in Core 1 cache, data B in Core 2 cache.
  And data A, data B has this happen before relationship.
  i.e
  In Core 1, update data A(Core 1), then update data B(Core 2).
  In Core 2, read data A(Core 1), verify data B(Core 2).

 
You see the problem. Between operation on A/B there's interleaves.
 
Core 1 updates data B, write into store buffer, send read invalidate to Core 2 on data B, at the same time, Core 2 reads data A, receives data A from Core 1, and verify data B as well, at the time, data B isn't invalidated yet.

Thus we need some wait mechanism.
i.e memory barrier.

In the above case, we need

• WMB(write memory barrier) on Core 1,
• RMB(read memory barrier) on Core 2. 

For Core 1, we do:
Update B, then WMB, then update data A.

For Core 2, we do:
Read data A, if A is old data, verify B won't happen, and if A is new data.
i.e
Core 1 updated data A, data B in Core 2 SHOULD use new one from Core 1, since it's invalidate queued.
But how to trigger Core 2 to verify invalidate queue? Before verify data B in Core 2, call RMB.
So the sequence for Core 2 will become:
Read Data A, RMB, verify data B.
     
 
WMB:

• Send read invalidate to Core 2 and till Core 2 send back invalidate ack will Core 1 flush data A from store buffer to it's cache line.

RMW:

• Verify invalidate queue, make data that is in the invalidate queue invalid the Core 2's cache line.
• While Core 2 tries to read the data, it will send a 'read' request to Core 1 for data B.

Take away:

• WMB should be issued AFTER a write to the shared data.
• RMW should be issued BEFORE a read to the shared data.

     
Reference:

1. Memory Barriers: a Hardware View for Software Hackers
2. https://en.cppreference.com/w/cpp/atomic/memory_order
3. https://en.cppreference.com/w/cpp/atomic/atomic_thread_fence

Code:

#include <atomic>
#include <iostream>
#include <thread>

using namespace std;
atomic<int> A{0};
atomic<int> B{0};

void t1()
{
    this_thread::sleep_for(1s);
    B.store(38, memory_order_relaxed);

    int a = 42;
    // WMB, prevents 'a' passing A.store.
    A.store(a, memory_order_release);
}

void t2()
{
    // RMB
    while (A.load(memory_order_acquire) != 42) {
        cout << "in while" << endl;
        // Prints 0 or 38 if B.store hasn't process yet.
        cout << B.load(memory_order_relaxed) << endl;
    }
    cout << "out while" << endl;
    // Always print 38.
    cout << B << endl;
}
int main()
{
    thread T1{t1};
    thread T2{t2};
    T1.join();
    T2.join();
}

Fence:

#include <atomic>
#include <iostream>
#include <thread>
using namespace std;
atomic<int> A{0};
atomic<int> B{0};

void t1()
{
    A.store(42, memory_order_relaxed);
    atomic_thread_fence(memory_order_release);
    // B.store can NEVER before A.store.
    B.store(38, memory_order_relaxed);
}

void t2()
{
    while (B.load(memory_order_relaxed) == 38) {
        cout << "in while loop\n";
        cout << A << endl; // Must be 42
        break;
    }
    cout << "out while loop\n";
}

int main()
{
    thread T1{t1};
    thread T2{t2};
    T1.join();
    T2.join();
}

Fence:

Release/Store

• Prevents all preceding memory operations from being reordered past subsequent writes.
• Prevents all following memory operations from being memory reordered before the write.

Acquire/Load

• Prevents all following memory operations from being reordered before this fence.
• Prevents all preceding memory operations from being reordered pass this fence.

Memory fences are NOT an acquire or release operation.

Operation:

Release operation:  store

• Cannot be reordered by compiler.
• Prevents preceding memory operations from being reordered past itself.
  i.e Any operations after a store can be reordered before the store operation.
• Any read or write operation that precedes it in program order.
  i.e those in memory_order_relaxed mode can't be reordered.

     
Acquire operation:  load

• https://en.cppreference.com/w/cpp/atomic/atomic_load
• Cannot be reordered by compiler.
• Any read or write operation that follows it in program order.
  i.e those in memory_order_relaxed mode can't be reordered.
• Those before the load can be memory ordered pass the load. Not as strong as fence.

BUT with different variable(object):
A release operation followed by a acquire operation CAN be reordered.
A acquire operation followed by a release operation CAN be reordered.
i.e
    A.store(1, std::memory_order_release);
    int b = B.load(std::memory_order_acquire);
=>
    int b = B.load(std::memory_order_acquire);
    A.store(1, std::memory_order_release);

However, keep in mind that (different variable/object) even reorder is OK for release/acquire operation, standard also depicts:
http://eel.is/c++draft/intro.multithread#intro.progress-18
An implementation should ensure that the last value (in modification order) assigned by an atomic or synchronization operation will become visible to all other threads in a finite period of time.

Reference:

https://stackoverflow.com/questions/14687703/what-c11-atomic-operations-memory-orders-guarantees-freshness

https://stackoverflow.com/questions/8819095/concurrency-atomic-and-volatile-in-c11-memory-model/8833218#8833218

i.e
If an store operation follows a for/while loop and a load operation,
the compiler shouldn't reorder load before store due to it can't
reasoning that the for/while loop is finite thus other thread can see thread store result in a finite time.
   
Reference:

1. [Preshing] Can Reordering of Release/Acquire Operations Introduce Deadlock?
2. [Bruce Dawson] In Praise of Idleness
3. [Golang bug list] cmd/compile: go1.8 regression: sync/atomic loop elided #19182

Hardware Memory Model:

Weak memory model

• ARM/Power PC

Strong memory model

• x86/64
• Sequential Consistence (software)

Reordering:

Sequentially Consistent types:

• Java: volatile variables
• C++11: atomic (default)

Explicit Compiler Barriers:
gcc:
    asm volatile("" ::: "memory");
Macro:
    #define COMPILER_BARRIER() asm volatile("" ::: "memory")

source:
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/barrier.h#L22

figure source: (https://www.youtube.com/watch?v=ZQFzMfHIxng)

Don't consider Sequential Consistency is SLOW; correctness is the highest priority.

Implied Compiler Barriers:
C++11:

• Every non­-relaxed atomic operation acts as a compiler barrier.
  這裡是compiler reordering, 不是memory reordering.
• Every function containing a compiler barrier must act as a compiler barrier itself, even the function is inlined.
• The majority of function calls act as compiler barriers, whether they contain their own compiler barrier or not due to it's coming from different TU. 
• But does not include inline functions, functions declared with the pure attribute, and cases where link ­time code generation is used.          

Reference:

1. [Implications of pure and constant functions] https://lwn.net/Articles/285332/

What about Golang?

goroutine act as 'user space thread', 

i.e it's runtime memory location is allocated on the heap.

Inside a single goroutine, the compiler is allowed to re-arrange

expressions as long as the reordering does not change the behavior within that goroutine as defined by the language specification. 

Within a single goroutine, the happens-before order is the order expressed by the program.

i.e

a := 42
if a == 42 {
    // always true.
}

// b will always be 42
b := a
// within other goroutines, the observe of a == 42 and c == 38 sequence is not guaranteed.
c := 38  

Reads and writes of values larger than a single machine word behave as multiple machine-word-sized operations in an unspecified order.

Initialization:

• If a package p imports package q, the completion of q's init functions happens before the start of any of p's.
• The start of the function main.main happens after all init functions have finished.

Goroutine creation:

• The go statement that starts a new goroutine happens before the goroutine's execution begins.
• i.e go statement act as a barrier function call in C/C++ which won't be reordered.

Goroutine destruction:

• Can happen any time.
• Be ware that if a goroutine that updates a global value and do nothing (i.e using channel etc.), an aggressive compiler could delete the goroutine entirely for optimization.
  i.e just modify the global variable without creating a goroutine.
  https://github.com/golang/go/issues/19182

Channel communication:

• A send on a channel happens before the corresponding receive from that channel completes.
• The closing of a channel happens before a receive that returns a zero value because the channel is closed.
• A receive from an un-buffered channel happens before the send on that channel completes.
• The kth receive on a channel with capacity C happens before the k+Cth send from that channel completes.

i.e the famous:
Do not communicate by sharing memory; instead, share memory by communicating.

No more memory store/load operation from programmer's point of view.

The Channel acted as the sequential consistence/memory fence contract.

Reference:

https://golang.org/ref/mem

https://blog.golang.org/share-memory-by-communicating

https://github.com/8l/go-learn

https://dougrichardson.org/2016/01/23/go-memory-allocations.html

[Preventing stack guard-page hopping] https://lwn.net/Articles/725832/

[C++] std::optional use idiom [Sum type]

article:
https://blogs.msdn.microsoft.com/vcblog/2018/09/04/stdoptional-how-when-and-why/


std::optional
std::any
std::variant

using T = /* some object type */;

struct S {
  optional<T> maybe_T;    

  void construct_the_T(int arg) {
    // We need not guard against repeat initialization;
    // optional's emplace member will destroy any 
    // contained object and make a fresh one.        
    maybe_T.emplace(arg);
  }

  T& get_the_T() { 
    assert(maybe_T);
    return *maybe_T;    
    // Or, if we prefer an exception when maybe_T is not initialized:
    // return maybe_T.value();
  }

  // ... No error-prone handwritten special member functions! ...
};

Reference:
https://vsdmars.blogspot.com/2018/05/caccu2018-tricks-library-implementation.html
https://vsdmars.blogspot.com/2018/06/c-regular-type.html

golang:
https://godoc.org/google.golang.org/cloud/internal/optional
src:
https://github.com/GoogleCloudPlatform/google-cloud-go/blob/master/internal/optional/optional.go

Java:
https://docs.oracle.com/javase/8/docs/api/java/util/Optional.html

[C++][Papers]

• Loop Recognition in C++/Java/Go/Scala