Edit model card

i2b2 QueryBuilder - 34b Merged

Screenshot

Model Description

This model will generate queries for your i2b2 query builder trained on this dataset for 10 epochs . For evaluation use.

  • Do not use as a final research query builder.
  • Results may be incorrect or mal-formatted.
  • The onus of research accuracy is on the researcher, not the AI model.

Prompt Format

If you are using text-generation-webui, you can download the instruction template i2b2.yaml

Below is an instruction that describes a task.

### Instruction:
{input}

### Response:
```xml

Architecture

nmitchko/i2b2-querybuilder-34b-merged is a large language model LoRa specifically fine-tuned for generating queries in the i2b2 query builder. It is based on codellama-34b-hf at 34 billion parameters.

The primary goal of this model is to improve research accuracy with the i2b2 tool. It was trained using LoRA, specifically QLora Multi GPU, to reduce memory footprint.

See Training Parameters for more info This Lora supports 4-bit and 8-bit modes.

Requirements

bitsandbytes>=0.41.0
peft@main
transformers@main

Steps to load this model:

  1. Load base model (codellama-34b-hf) using transformers

Training Parameters

The model was trained for or 10 epochs on i2b2-query-data-1.0 i2b2-query-data-1.0 contains only tasks and outputs for i2b2 queries xsd schemas.

Item Amount Units
LoRA Rank 64 ~
LoRA Alpha 16 ~
Learning Rate 1e-4 SI
Dropout 5 %

Training procedure

The following bitsandbytes quantization config was used during training:

  • quant_method: QuantizationMethod.BITS_AND_BYTES
  • load_in_8bit: False
  • load_in_4bit: True
  • llm_int8_threshold: 6.0
  • llm_int8_skip_modules: None
  • llm_int8_enable_fp32_cpu_offload: False
  • llm_int8_has_fp16_weight: False
  • bnb_4bit_quant_type: nf4
  • bnb_4bit_use_double_quant: True
  • bnb_4bit_compute_dtype: bfloat16

Framework versions

  • PEFT 0.6.0.dev0
Downloads last month
0
Safetensors
Model size
33.7B params
Tensor type
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train nmitchko/i2b2-querybuilder-34b-merged