Programming notesProgramming notes Hi-Tech |
Comenteaza
Migrating a project from Java 11 to Java 17
Java 17 is new LTS version and it make sense to migrate existing projects to the new version. I will describe my specific case
1) So first I updated maven pom.xml and put there - <java.version>17</java.version>2) Next I updated docker image - to this one - openjdk:173) In dependencies I have spring boot so updated it to version 2.6.8 4) There were some difficulties with Ignite Cache, we should use previous version that uses some features from previous jdk that why I added this vm options to open access to closed packets:
org.apache.maven.plugins
maven-surefire-plugin
3.0.0-M7
--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
--add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
--add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
--add-opens=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED
--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens java.base/java.time=ALL-UNNAMED
5) We need to add the same options in java opts when we starting our application, we use docker how i said so i created JAVA_OPTS vars with props above, and added to docker run
ENTRYPOINT java $JAVA_OPTS -jar super-project.jar
6) Next there were some issues with swagger that why I updated to springdoc-openapi-ui 1.6.117) I've spent a lot of time to resolve issue with Jaeger tracing, it was disabled locally and everything was ok but when we start our app on server we have exceptions. So we decided to move from jaeger to spring-cloud-starter-sleuth - 2021.0.2 In general, these are the most notable bugs I encountered during the migration to Java 17 which I found useful to mention. So it is not scary at all)
Generate settings-security.xml file for maven password encryption
mvn --encrypt-master-password does not create a file security-settings.xml. Use the command mvn --encrypt-master-password to generate a master password. Remember this password.Create a new file security-settings.xml in ${user.home}/.m2/. For example, on Mac OS X or Ubuntu as ~/.m2/settings-security.xml.Write into this file:<settingsSecurity>
<master>OUTPUT OF THE COMMAND: mvn --encrypt-master-password</master>
</settingsSecurity>e.g.:<settingsSecurity>
<master>{jSMOWnoPFgsHVpMvz5VrIt5kRbzGpI8u+9EF1iFQyJQ=}</master>
</settingsSecurity>After that the maven encrypt command works on the command line:mvn --encrypt-passwordAs mentioned by khmarbaise, more detailed information can be found here: https://maven.apache.org/guides/mini/guide-encryption.htmlsource
Intellij Idea can't resolve generated class
It was really annoying issue, we had some maven plugin that generates classes from some schema, and for some reason Intellij Idea cant resolve this class. It just marks it with red. The reason was too big file size i've increased it like this:
# Maximum file size (kilobytes) IDE should provide code assistance for.
idea.max.intellisense.filesize=6000
# Maximum file size (kilobytes) IDE is able to open.
idea.max.content.load.filesize=6000
in Help > Edit Custom Properties
and then File > Restart & invalidate
End-to-end tests with Spring and Test Containers
I've heard previously about test-containers but didn't have chance to read about it. Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
So basically it simplify you process of creating and maintaining end-to-end tests. You no need external containers you simple can run in on jenkins during your usual testing life cycle.
Two articles illustrate this approach with Selenium and one more with WireMock
Micrometer - SLF4J but for metrics
Nice tool to try, Micrometer is - facade over the instrumentation clients for a number of popular monitoring systems. Currently, it supports the following monitoring systems: Atlas, Datadog, Graphite, Ganglia, Influx, JMX and Prometheus. More in tutorial
Syntaxis is simple and straightforward:
Metrics.counter("objects.instance").increment();
Spring Data and MongoDb aggregation examples
I've faced recently with some not trivial mongo aggregation queries and found there are not so many examples available, so main source is official docs https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#introduction
also was useful tests of the Spring Framework:
https://github.com/spring-projects/spring-data-mongodb/blob/master/spring-data-mongodb/src/test/java/org/springframework/data/mongodb/core/aggregation/AggregationTests.java
https://github.com/spring-projects/spring-data-mongodb/blob/master/spring-data-mongodb/src/test/java/org/springframework/data/mongodb/core/aggregation/ProjectionOperationUnitTests.java
use to profile query
db.events.explain("executionStats").aggregate("query")
Useful Java tools to try
java mission control - to monitor jvm
lincheck - testing concurrent data structures
jmh - it is tool for microbenchmarking your code
debezium - tool to capture database changes (mongo, oracle...)
Spring Cloud Contract
Cool tool to test interaction between you apps without running all of them. Need just describe contract of you service and inject in via wiremock were it is needed. Here is a tutorial
Spring reactive
Spring introduced reactive support with WebFlux it implements basic Reactive concepts. It works with Netty, non blocking server.
Also Spring added support of the RSocket protocol it is not based on HTTP and provides nice features as back-pressure. You can establish direct connection from the web-browser to rsocket server. Spring provides API to implement listeners, smth similar like rest controllers.
Spring Boot 2 and custom JsonSerializer
I was migrating recently to the new Spring Boot version 2. And most annoying thing was that PageImpl (component used for pagination) changed format. Also I had some issues with Mockito 2, but all of them could be solved read here.
Well I found this article and it all looked easy despite the fact that Spring did not want to use my JsonSerializer. Here how it looks:
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import org.springframework.boot.jackson.JsonComponent;
import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.Sort;
import java.io.IOException;
@JsonComponent
public class PageImplJacksonSerializer extends JsonSerializer {
@Override
public void serialize(PageImpl page, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
jsonGenerator.writeStartObject();
jsonGenerator.writeObjectField("content", page.getContent());
jsonGenerator.writeBooleanField("first", page.isFirst());
jsonGenerator.writeBooleanField("last", page.isLast());
jsonGenerator.writeNumberField("totalPages", page.getTotalPages());
jsonGenerator.writeNumberField("totalElements", page.getTotalElements());
jsonGenerator.writeNumberField("numberOfElements", page.getNumberOfElements());
jsonGenerator.writeNumberField("size", page.getSize());
jsonGenerator.writeNumberField("number", page.getNumber());
Sort sort = page.getSort();
jsonGenerator.writeArrayFieldStart("sort");
for (Sort.Order order : sort) {
jsonGenerator.writeStartObject();
jsonGenerator.writeStringField("property", order.getProperty());
jsonGenerator.writeStringField("direction", order.getDirection().name());
jsonGenerator.writeBooleanField("ignoreCase", order.isIgnoreCase());
jsonGenerator.writeStringField("nullHandling", order.getNullHandling().name());
jsonGenerator.writeEndObject();
}
jsonGenerator.writeEndArray();
jsonGenerator.writeEndObject();
}
}
Next I read Spring docs of how can I customize Serializer here and here
They suggested to create jacksonBuilder with your specific module or register JsonComponentModule that will find your serializer that will be used by spring during mapper creation. Unfortunately none of this worked for me:
@Bean
public Module jsonComponentModule() {
return new JsonComponentModule();
}
And this also didn't work:
@Bean
@Primary
public Jackson2ObjectMapperBuilder jacksonBuilder() {
JsonComponentModule module = new JsonComponentModule();
module.addSerializer(PageImpl.class, new PageImplJacksonSerializer());
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.modulesToInstall(module);
return builder;
}
I continued with my last idea, suggested by docs is configuring HttpMessageConverter.
And here I figured out that my config extends WebMvcConfigurationSupport to enable MVC and
after debugging spring context creation. I found that Spring registers default converters with method
addDefaultHttpMessageConverters and following line of code and for some reason it does not register serializer here:
messageConverters.add(new MappingJackson2HttpMessageConverter(builder.build()));
So here is working solution for me, with configuring default converter created by Spring:
@Override
protected void extendMessageConverters(List<HttpMessageConverter<?>> converters) {
MappingJackson2HttpMessageConverter converter = (MappingJackson2HttpMessageConverter)
converters.stream()
.filter(c -> c instanceof MappingJackson2HttpMessageConverter)
.findFirst().get();
Jackson2ObjectMapperBuilder builder = Jackson2ObjectMapperBuilder.json();
JsonComponentModule module = new JsonComponentModule();
module.addSerializer(PageImpl.class, new PageImplJacksonSerializer());
ObjectMapper objectMapper = builder.modules(module).build();
converter.setObjectMapper(objectMapper);
}
Spring Boot articles to read
https://www.e4developer.com/2018/08/06/spring-boot-best-practices/
https://www.e4developer.com/2018/03/30/introduction-to-concurrency-in-spring-boot/
https://www.e4developer.com/2018/04/28/springs-webflux-reactor-parallelism-and-backpressure/
JVM benchmark example
A JVM benchmark example from real project (QuickFIX/J is library that implements finacial FIX protocol) https://github.com/quickfix-j/quickfixj/wiki/JMH-benchmark-for-%2339-and-%2342
Log4j not printing the stacktrace for exception
I've noticed interesting behavior in the server logs on jdk 8. There was an exception without stack trace. After small research I have found that hotspot may apply some optimization to exceptions that are thrown too frequent, it simply trim stack trace) So I scrolled to the first occurrences of the exception and found my stack trace. Here is oracle docs describing it:
http://www.oracle.com/technetwork/java/javase/relnotes-139183.html
and you can disable it with -XX:-OmitStackTraceInFastThrow
Spring 5 and HSQLDB sample app
Recently I have played a bit with Spring 5 and embedded database HSQLDB. It is very not usual for me to create Spring app even without single xml. Also I used cool library called Lombok it provides auto getter/setter generation.
What was interesting for me that spring-boot auto configure for you following features: mvc, transactions, json serialization, jpa, tests. It has even embedded tomcat)
Here is my github repository
Java 8 Run-Time Evaluation of Lambda Expressions
http://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.27.4
Java 8 Lambdas - A Peek Under the Hood
Machine learning
Choosing the right estimator
http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html
https://www.dropbox.com/s/nz2d6uquwn59fz6/ml_map.png
later i will add here more machine learning stuff
Первые шаги в машинном обучении
Machine Learning is Fun! The world’s easiest introduction to Machine Learning
https://oracle.github.io/graphpipe/#/
https://github.com/neomatrix369/awesome-ai-ml-dl
A different object with the same identifier value was already associated with the session
Hibernate Error: org.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session
What is the reason of such exception?
Session already contains object that you are trying to put there one more time with such identifier (for example with update). It also could be many-to-one or one-to-many relationship object or cascading operation.
How to deal with it?
probably you already called update() for this object, and trying do this 2nd time
another possible reason you making copy of the object that is already associated with session (has assigned id and was loaded recently) and trying to update it
try session.merge() - it will copy the state of the given object into the persistent object with the same identifier.
try session.evict(object) - remove this instance from the session cache.
Domain Driven Design in Java
I think most enterprise java world is suffering from Anemic Domain Model. One of the ways that can help you is DDD. I've started researching DDD area, and I've found couple of good books and sources for them:
Implementing Domain-Driven Design by Vaughn Vernon (I think it is good to start book)
and source code is here : https://github.com/VaughnVernon/IDDD_Samples
And one more book called Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans
and source code is here: https://github.com/citerus/dddsample-core
more code samples https://github.com/ddd-by-examples/library
https://github.com/mploed/ddd-with-spring
Free Java hosting options and Heroku tomcat deployment
I wanted to write some java app that would communicate with 3rd party API. The main problem there was that this API requires https connection. So I needed free java hosting to test my concept.
I will describe my short research about free java hosting proposals. And will describe shortly Heroku quick start.
My main options were:
Amazon AWS - it is free for one year. It has EC2 - it is linux virtual machine solution and Elastic Beanstalk - it is like a pass with command line to interact. The main disadvanges for me that it doesn't have free SSL.
Digital Ocean - it is not free but gives you Linux virtual machine.
Cloudflare - PaaS, it has SSL and it is free. Unfortunately didn't have chance to try it.
Openshift - it is PaaS, i tried it couple of years ago and it was too complicated for simple apps.
Heroku - it is PaaS that has SSL support. What is most optimal option for me. Main disadvantage could be that it is idling your app each 30 minutes if it is inactive, but it was ok for me.
So here is few steps that need to be done to deploy you java app on Heroku tomcat server:
1) Need to install Heroku CLI it includes Git
2) need to create maven project with Procfile file in the root, it should contain:
web: java $JAVA_OPTS -jar target/dependency/webapp-runner.jar --port $PORT target/*.war
3) Commit your chages to Git
git init
git add .
git commit -m "Ready to deploy"
4) Crete the app and & remote Git repo (of course login first with heroku login)
heroku create
5) Deploy your code
git push heroku master
6) Open your app url
heroku open
Voila)
Also useful commands:
To show logs
heroku logs --tail
To enable scalling
heroku ps:scale web=1
To open bash console
heroku run bash
To see addons, add databse
heroku addons
To see config vars
heroku config
Resources:
Getting Started on Heroku with Java
Deploying Tomcat-based Java Web Applications with Webapp Runner
The top of most mentioned books on stackoverflow.com
Top 5:
#1 Working Effectively with Legacy Code
Michael C. Feathers
#2 Design Patterns
Ralph Johnson, Erich Gamma, John Vlissides, Richard Helm
#3 Clean Code
Robert C. Martin
#4 Java concurrency in practice
Brian Goetz, Tim Peierls
#5 Domain-driven Design
Eric Evans
See more here http://www.dev-books.com/
Spring Boot: A Quick Start
Hmm spring configuration is getting better)
https://dzone.com/articles/spring-boot-a-quick-start
Scrum in few words
Note about scrum. It is mostly based on Scum Guide but in much shorter way and with some examples from my experience.
Scrum is a framework for developing and sustaining complex products
Roles
The Product Owner - is the person responsible for managing the Product Backlog (describing items, ordering and prioritization ). Also Product owner may delegate his responsibilities to the Dev Team.
However, the
Product Owner remains accountable, and you should consult with him.
The Development Team - decide how to turn Product Backlog into Increments of potentially releasable functionality. Are cross-functional, with all of the skills to create
a product Increment;
Usually having 3-9 members.
The Scrum Master -is responsible for ensuring Scrum is understood and enacted.
Service to the Product Owner (help in Product Backlog management, helping the Scrum Team understand product backlog items, Facilitating Scrum events)
Service to the Development Team (Coaching, Helping the Development Team to create product, Facilitating Scrum events, Removing impediments)
Scrum Events
The heart of Scrum is a Sprint, a time-box of one month or less during which a “Done”, usable,
and potentially releasable product Increment is created. Sprints best have consistent duration
throughout a development effort. A new Sprint starts immediately after the conclusion of the
previous Sprint.
Sprints contain and consist of the Sprint Planning, Daily Scrums, the development work, the
Sprint Review, and the Sprint Retrospective.
Sprint Planning
Sprint Planning answers the following:
What can be delivered in the Increment resulting from the upcoming Sprint?
How will the work needed to deliver the Increment be achieved?
The input to this meeting is the Product Backlog, the latest product Increment, projected
capacity of the Development Team during the Sprint, and past performance of the Development
Team. The number of items selected from the Product Backlog for the Sprint is solely up to the
Development Team. Only the Development Team can assess what it can accomplish over the
upcoming Sprint.
The Development Team usually starts by designing the system and the work needed to convert
the Product Backlog into a working product Increment.
The Product Owner can help to clarify the selected Product Backlog items and make trade-offs.
If the Development Team determines it has too much or too little work, it may renegotiate the
selected Product Backlog items with the Product Owner.
For example each member says his capacity. Scrum master displays all jiras with some release version. All team estimates jiras. You could also use excell to group planned tasks.
Frequent approach is sending email with sprint backlogs items after planning to Product Owner (if he can't participate)
Sprint Goal
The Sprint Goal is an objective set for the Sprint that can be met through the implementation of
Product Backlog. It provides guidance to the Development Team on why it is building the
Increment.
Try to make it clear, you could use SMART approach
Daily Scrum
The Daily Scrum is a 15-minute time-boxed event for the Development Team to synchronize
activities and create a plan for the next 24 hours. This is done by inspecting the work since the
last Daily Scrum and forecasting the work that could be done before the next one.
the Development Team members explain:
What did I do yesterday that helped the Development Team meet the Sprint Goal?
What will I do today to help the Development Team meet the Sprint Goal?
Do I see any impediment that prevents me or the Development Team from meeting the
Sprint Goal?
Very useful is tracking progress of each individual task (maybe visually), tracking time (ex member A done 2h from 15h ), use color markers for each task (red/green/blue approach depending on item, status).
It could be on scrum board with three main parts todo/ in progress/ done.
Sprint Review
A Sprint Review is held at the end of the Sprint to inspect the Increment and adapt the Product
Backlog if needed.During the Sprint Review, the Scrum Team and stakeholders, product owner collaborate
about what was done in the Sprint.The entire group collaborates on what to do next.
The result of the Sprint Review is a revised Product Backlog
For example demo of new features is good thing here.
Sprint Retrospective
The Sprint Retrospective is an opportunity for the Scrum Team to inspect itself and create a plan
for improvements to be enacted during the next Sprint. Inspect how the last Sprint went with regards to people, relationships, process,and tools.
Good approach here to write on board pluses and minuses and possible todo plan to implement in new sprint.
Scrum Artifacts
Product Backlog - is an ordered list of everything that might be needed in the product.
Good technique sometimes is to refine some big or complex product backlog items for the next sprint, to be able to estimate them well on the planning
Sprint Backlog - is the set of Product Backlog items selected for the Sprint, plus a plan for
delivering the product Increment and realizing the Sprint Goal. It is only forecast.
Good technique is to add label in jira with sprint number
Definition of “Done” - when a Product Backlog item or an Increment is described as “Done”, everyone must
understand what “Done” means.
For example developer committed change and closed an jira, QA passed jira, Reporter closed it.
Good technique is to have it printed somewhere to be visible for all team.
References
Download the official Scrum Guide
Stuff Every Agile Development Team Needs to Know: A Primer
Spring Transactions management tips
In this article I will describe ways of managing transactions in Spring. And will share some useful notes based on my experience
Here is two ways to manage transactions in spring:
1) Programmatic transaction with TransactionTemplate class
this means that you have to manage the transaction with the help of programming. That gives you flexibility, but it is difficult to maintain.
-here is typical example:
TransactionTemplate txTemplate = new TransactionTemplate(txManager);
txTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
txTemplate.execute(new TransactionCallback<object>() {
public Object doInTransaction(TransactionStatus status) {
// do stuff
}
});
-be aware that A RuntimeException thrown by the callback is treated as a fatal exception that enforces a rollback. Such an exception gets propagated to the caller of the template (see TransactionTemplate source in references).
2) Declarative transaction with @Transactional annotation
this means you separate transaction management from the business code. You only use annotations or XML-based configuration to manage the transactions
-default propagation is Propagation.REQUIRED
-default configured to rollback just Unchecked exceptions (ex RuntimeException), to rollback Exception need to use it like this:
@Transactional(rollbackFor = Exception.class)
-also method should be public, spring will wrap your method with proxy on context start and
will add commit/rollback logic, that is how transnational annotation magic works.
-call to transnational method should be made from different class not from the same class,
it is limitation of CGLIB (library that is doing method wrapping by the way you can
use AspectJ)
-exception saying that "the transaction was marked as rollback only" it is because setRollbackOnly() method was called by exception (by the way you can call it too using previous approach), you can prevent it like this, or start new transaction for each part that should be handled independently
@Transactional(rollbackFor=MyException.class, noRollbackFor=MyException2.class)
-the Service is the best place for putting @Transactional
References:
16. Transaction Management
TransactionTemplate.java source code
Is Inheritance Dead? A Detailed Look Into the Decorator Pattern
https://dzone.com/articles/is-inheritance-dead
Generat în 0.507 secunde.