序言
JDK序列化原理分析
序列化整体流程
private void writeObject(java.io.ObjectOutputStream s)
throws java.io.IOException {
java.io.ObjectOutputStream.PutField fields = s.putFields();
fields.put("rnd", U.getLong(Thread.currentThread(), SEED));
fields.put("initialized", true);
s.writeFields();
}
需要注意defaultWriteObject写的数据可能会通过readFields进行读取,因此其格式需要和和putFields兼容。另外在自定义序列化时defaultWriteObject/putFields两者只能调用一个。
反序列化整体流程
Hessian/Kryo等框架存在的问题
Hessian存在的问题
public static class CustomReplaceClass implements Serializable {
Object writeReplace() {
return new CustomReplaceClass();
}
Object readResolve() {
return new CustomReplaceClass();
}
}
Exception in thread "main" java.lang.StackOverflowError
at java.base/java.lang.reflect.InvocationTargetException.<init>(InvocationTargetException.java:73)
at jdk.internal.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.caucho.hessian.io.WriteReplaceSerializer.writeReplace(WriteReplaceSerializer.java:184)
at com.caucho.hessian.io.WriteReplaceSerializer.writeObject(WriteReplaceSerializer.java:155)
at com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:465)
at com.caucho.hessian.io.WriteReplaceSerializer.writeObject(WriteReplaceSerializer.java:167)
at com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:465)
at com.caucho.hessian.io.WriteReplaceSerializer.writeObject(WriteReplaceSerializer.java:167)
at com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:465)
at com.caucho.hessian.io.WriteReplaceSerializer.writeObject(WriteReplaceSerializer.java:167)
at com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:465)
/**
* Head of linked list.
* Invariant: head.item == null
*/
transient Node<E> head;
/**
* Tail of linked list.
* Invariant: last.next == null
*/
private transient Node<E> last;
private void writeObject(java.io.ObjectOutputStream s)
throws java.io.IOException {
fullyLock();
try {
// Write out any hidden stuff, plus capacity
s.defaultWriteObject();
// Write out all elements in the proper order.
for (Node<E> p = head.next; p != null; p = p.next)
s.writeObject(p.item);
// Use trailing null as sentinel
s.writeObject(null);
} finally {
fullyUnlock();
}
}
private void readObject(java.io.ObjectInputStream s)
throws java.io.IOException, ClassNotFoundException {
s.defaultReadObject();
readHolds = new ThreadLocalHoldCounter();
setState(0); // reset to unlocked state
}
Kryo存在的问题
JDK序列化性能很差,导致kryo序列化性能大幅退化
JDK序列化结果很大,导致kryo序列化数据膨胀
转发给JDK序列化的对象子图不会跟Kryo share同一个引用表,如果该子图共享/循环引用了其它对象,则会出现重复序列化/递归栈溢出
kryo不支持父子类出现重名字段,这在某些条件下也会成为一个使用限制。
其它框架存在的问题
Jsonb不支持任何JDK自定义序列化方法,反序列化会报错
Fury兼容实现原理
早期序列化流程
新版序列化流程
整体实现在:
io.fury.serializers.ReplaceResolveSerializer和
io.fury.serializers.ObjectStreamSerializer
ObjectStreamSerializer实现了整套:
JDK writeObject/readObject/readObjectNoData/registerValidation
行为,保证行为跟JDK的一致性,在任意情况下序列化都不会报错。
由于用户在:
writeObject/readObject/ readObjectNoData/registerValidation
里面调用的是:
JDK ObjectOutputStream/ObjectInputStream /PutField/GetField的接口
因此Fury也实现了一套:
ObjectOutputStream/ObjectInputStream/PutField/ GetField
的子类,保证实际序列化逻辑可以转发给Fury。
为了保证类型前后兼容,同时保证:
defaultWriteObject/defaultReadObject
创建JITCompatibleSerializer进行序列化。
Constructor constructor;
try {
constructor = type.getConstructor();
if (!constructor.isAccessible()) {
constructor.setAccessible(true);
}
} catch (Exception e) {
constructor =
(Constructor) ReflectionUtils.getObjectFieldValue(ObjectStreamClass.lookup(type), "cons");
}
List<SlotsInfo> slotsInfoList = new ArrayList<>();
Class<?> end = type;
// locate closest non-serializable superclass
while (end != null && Serializable.class.isAssignableFrom(end)) {
end = end.getSuperclass();
}
while (type != end) {
slotsInfoList.add(new SlotsInfo(fury, type));
type = type.getSuperclass();
}
Collections.reverse(slotsInfoList);
slotsInfos = slotsInfoList.toArray(new SlotsInfo[0]);
private static class SlotsInfo {
private final Class<?> cls;
private final Method writeObjectMethod;
private final Method readObjectMethod;
private final Method readObjectNoData;
private final CompatibleSerializerBase slotsSerializer;
private final ObjectIntMap<String> fieldIndexMap;
private final FieldResolver putFieldsResolver;
private final CompatibleSerializer compatibleStreamSerializer;
private final FuryObjectOutputStream objectOutputStream;
private final FuryObjectInputStream objectInputStream;
private final ObjectArray getFieldPool;
}
序列化执行部分
写入所有Serializable class数量
遍历对象类层次结构,依次序列化每个类型的字段数据。序列化每个类型的数据分为以下几个部分:
如果当前对象所在类型没有定义writeObject方法,则直接调用slotsSerializer (JITCompatibleSerializer)序列化当前类型所有字段。
如果前对象所在类型定义了writeObject方法,则会缓存之前一次序列化的上下文,然后调用writeObject方法,传入Fury实现的FuryObjectOutputStream。
for (SlotsInfo slotsInfo : slotsInfos) {
buffer.writeShort((short) slotsInfos.length);
classResolver.writeClassInternal(buffer, slotsInfo.cls);
Method writeObjectMethod = slotsInfo.writeObjectMethod;
if (writeObjectMethod == null) {
slotsInfo.slotsSerializer.write(buffer, value);
} else {
FuryObjectOutputStream objectOutputStream = slotsInfo.objectOutputStream;
Object oldObject = objectOutputStream.targetObject;
MemoryBuffer oldBuffer = objectOutputStream.buffer;
FuryObjectOutputStream.PutFieldImpl oldPutField = objectOutputStream.curPut;
boolean fieldsWritten = objectOutputStream.fieldsWritten;
try {
objectOutputStream.targetObject = value;
objectOutputStream.buffer = buffer;
objectOutputStream.curPut = null;
objectOutputStream.fieldsWritten = false;
writeObjectMethod.invoke(value, objectOutputStream);
} finally {
objectOutputStream.targetObject = oldObject;
objectOutputStream.buffer = oldBuffer;
objectOutputStream.curPut = oldPutField;
objectOutputStream.fieldsWritten = fieldsWritten;
}
}
}
根据构造函数创建对象实例。
将对象实例写入引用表。
读取对象层次结构所有Serializable Class数量。
依次从数据读取class,并和当前类型层次结构的class进行比较。如果不一致,则代表当前类型层次结构发生了变化,引入了新的父类,如果该类型定义了readObjectNoData,则调用该方法进行初始化,然后向上遍历类型层次结构,直到找到相同类型。
反序列化该类型的所有字段值并设置到对象字段上面。
如果对象没有定义readObject方法,则直接调用slotsSerializer (JITCompatibleSerializer)进行反序列化。
如果定义了readObject方法,则调用对象的readObject方法,传入Fury实现的FuryObjectInputStream。
在FuryObjectInputStream里面,同时也会针对readFields/defaultReadObject进行特殊的处理。readFields会使用CompatibleSerializer把对象转换成可识别的GetField形式,defaultReadObject则会直接调用slotsSerializer (JITCompatibleSerializer)反序列化当前类型所有字段。
如果在readObject期间用户通过registerValidation注册了ObjectInputValidation回调,则会在返回该对象之前,按照优先级依次执行回调。
至此反序列化完成。核心代码大致如下:
Object obj = null;
if (constructor != null) {
try {
obj = constructor.newInstance();
} catch (InstantiationException | IllegalAccessException | InvocationTargetException e) {
Platform.throwException(e);
}
} else {
obj = Platform.newInstance(type);
}
fury.getReferenceResolver().reference(obj);
int numClasses = buffer.readShort();
int slotIndex = 0;
TreeMap<Integer, ObjectInputValidation> callbacks = new TreeMap<>(Collections.reverseOrder());
for (int i = 0; i < numClasses; i++) {
Class<?> currentClass = classResolver.readClassInternal(buffer);
SlotsInfo slotsInfo = slotsInfos[slotIndex++];
while (currentClass != slotsInfo.cls) {
// the receiver's version extends classes that are not extended by the sender's version.
Method readObjectNoData = slotsInfo.readObjectNoData;
if (readObjectNoData != null) {
readObjectNoData.invoke(obj);
}
slotsInfo = slotsInfos[slotIndex++];
}
Method readObjectMethod = slotsInfo.readObjectMethod;
if (readObjectMethod == null) {
slotsInfo.slotsSerializer.readAndSetFields(buffer, obj);
} else {
FuryObjectInputStream objectInputStream = slotsInfo.objectInputStream;
MemoryBuffer oldBuffer = objectInputStream.buffer;
Object oldObject = objectInputStream.targetObject;
FuryObjectInputStream.GetFieldImpl oldGetField = objectInputStream.getField;
FuryObjectInputStream.GetFieldImpl getField =
(FuryObjectInputStream.GetFieldImpl) slotsInfo.getFieldPool.popOrNull();
if (getField == null) {
getField = new FuryObjectInputStream.GetFieldImpl(slotsInfo);
}
boolean fieldsRead = objectInputStream.fieldsRead;
try {
objectInputStream.fieldsRead = false;
objectInputStream.buffer = buffer;
objectInputStream.targetObject = obj;
objectInputStream.getField = getField;
objectInputStream.callbacks = callbacks;
readObjectMethod.invoke(obj, objectInputStream);
} finally {
objectInputStream.fieldsRead = fieldsRead;
objectInputStream.buffer = oldBuffer;
objectInputStream.targetObject = oldObject;
objectInputStream.getField = oldGetField;
slotsInfo.getFieldPool.add(getField);
objectInputStream.callbacks = null;
Arrays.fill(getField.vals, FuryObjectInputStream.NO_VALUE_STUB);
}
}
}
for (ObjectInputValidation validation : callbacks.values()) {
validation.validateObject();
}
性能对比
结论
[1]writeObject:https://docs.oracle.com/en/java/javase/18/docs/specs/serialization/output.html#the-writeobject-method
[2]readObject:
https://docs.oracle.com/en/java/javase/18/docs/specs/serialization/input.html#the-readobject-method
[3]writeReplace:
https://docs.oracle.com/en/java/javase/18/docs/specs/serialization/output.html#the-writereplace-method
[4]readResolve:https://docs.oracle.com/en/java/javase/18/docs/specs/serialization/input.html#the-readresolve-method
Java应用提速(速度与激情)
本文将阐述java应用如何通过基础设施与工具的改进,实现从构建到启动全方面大幅提速,内容涵盖:maven构建提速、本地IDEA环境提速、docker构建提速、JDK提速、Classloader提速、阿里中间件提速以及其他提速。
点击阅读原文查看详情。