Skip to content

[Improvement] org.apache.spark.serializer.JavaSerializer does not support object relocation. #2278

Description

@maobaolong

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

What would you like to be improved?

24/10/24 21:14:00 ERROR ApplicationMaster: User class threw exception: java.lang.IllegalArgumentException: Can't use serialized shuffle for shuffleId: 0, because the serializer: org.apache.spark.serializer.JavaSerializer does not support object relocation.

How should we improve?

Support reject JavaSerializer earlier and use sort shuffleManager instead.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions