jstzwj
init weights
b69ed44

360LayoutAnalysis

github

Chinese

I. Background

In today's digital age, document layout analysis is one of the key steps in information extraction and document understanding. Document layout analysis, also known as document image analysis or document layout analysis, refers to the process of identifying and extracting text, images, tables, and other elements from scanned document images. This technology has extensive applications in fields such as automated document processing, electronic data exchange, and digitization of historical documents. Traditional document layout analysis models often find it difficult to accurately distinguish paragraphs and other layout elements in documents, which limits the further processing and utilization of document information. The development of deep learning and pattern recognition technologies has brought new opportunities for document layout analysis. By training datasets, the model's understanding of document structure can be improved. However, high-quality annotated datasets are the foundation for training effective models. In document layout analysis, refined annotation is very necessary, among which paragraph annotation is particularly crucial because it directly affects the semantic understanding and information extraction of the text. Currently, in the field of layout analysis, as far as we know, in paper scenarios, previous open-source datasets such as CDLA (A Chinese document layout analysis) lack annotation of paragraph information; The layout analysis model in the research report scenario is still relatively lacking. Therefore, in order to solve this problem, we manually annotated the paper documents for fine-grained label transformation and data optimization, and constructed a fine-grained layout analysis dataset for research report scenarios. It is best to use these annotated datasets to train multiple new Chinese document layout analysis models, which performed well on the closed test set. In this open source project, we have prioritized the development of lightweight model weights and corresponding label systems for page analysis in two scenarios: paper and research report. The aim is to identify paragraph boundaries and other information in documents, accurately distinguish text, images, tables, formulas, and other elements, and ultimately promote industrial development.

II. Usage

  • Weights download link: 🤗LINK

  • Usage:

    The open-source weights are trained with yolov8, and the prediction method is as follows:

    from ultralytics import YOLO
    
    image_path = ''  # Path to the image to be predicted
    model_path = ''  # Path to the weights
    model = YOLO(model_path)
    
    result = model(image_path, save=True, conf=0.5, save_crop=False, line_width=2)
    print(result)
    
    print(result[0].names)         # Output the id to label map
    print(result[0].boxes)         # Output all detected bounding boxes
    print(result[0].boxes.xyxy)    # Output the top-left and bottom-right coordinates of all detected bounding boxes
    print(result[0].boxes.cls)     # Output the id corresponding to the class of all detected bounding boxes
    print(result[0].boxes.conf)    # Output the confidence of all detected bounding boxes
    

III. Layout Analysis

3.1 Academic Paper Scenario

  • Label Categories

    Element Name
    Text Main Text (Paragraph)
    Title Title
    Figure Image
    Figure caption Image Caption
    Table Table
    Table caption Table Caption
    Header Header
    Footer Footer
    Reference Reference
    Equation Equation
  • Example

3.2 Research Report Scenario

  • Label Categories

    Element Name
    Text Main Text (Paragraph)
    Title Title
    Figure Image
    Figure caption Image Caption
    Table Table
    Table caption Table Caption
    Header Header
    Footer Footer
    Toc Table of Contents
  • Example

3.3 publaynet

  • Label Categories

    Element Name
    text Main Text (Paragraph)
    title Title
    figure Image
    list List
    table Table
  • Example

3.4 General Layout

  • Label category

    Element Name
    Text Main text
    Title Title
    Figure List
    Table Table
    Equation Image
    Caption Image Caption/Table Caption

License

This project utilizes certain datasets and checkpoints that are subject to their respective original licenses. Users must comply with all terms and conditions of these original licenses. The content of this project itself is licensed under the Apache license 2.0.

License

The source code of this repository follows the open-source license Apache 2.0. The 360LayoutAnalysis model open-source model supports commercial use. If you need to use this model and its derivative models for commercial purposes, please apply through the email ([email protected]), and see the specific license agreement in "360LayoutAnalysis Model Open Source Model License".