dkcodes/poly-headline

SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 2048 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dkcodes/poly-headline")
# Run inference
queries = [
    "Binance adds $SMOLE to its spot crypto exchange",
]
documents = [
    '$SMOLE listed on Binance in March? This market will resolve to "Yes" if the crypto token smolecoin ($SMOLE) is listed for spot purchase on Binance by March 31, 2024, 11:59 PM ET. Otherwise, this market will resolve to "No". The primary resolution source for this market will be Binance, however a consensus of credible reporting will also be used.',
    'Historical Overview: Binance’s Impact on Global Cryptocurrency Trading Since 2017',
    'FDA approves PTC Therapeutics’ Vatiquinone for Friedreich’s ataxia? This market will resolve to "Yes" if the U.S. Food and Drug Administration (FDA) grants full or conditional approval for PTC Therapeutics’ Vatiquinone as a treatment for Friedreich’s ataxia by August 31, 2025, 11:59 PM ET. Otherwise, this market will resolve to "No." An approval is defined as: For new drugs: FDA issuance of an approval letter for a New Drug Application (NDA) or Biologics License Application (BLA) For already-marketed drugs seeking new indications: FDA approval of a supplemental NDA (sNDA) or supplemental BLA (sBLA) for the specific indication referenced For generic drugs: FDA approval of an Abbreviated New Drug Application (ANDA) For biosimilars: FDA approval of a 351(k) application The following constitute qualifying approvals: Standard approval (traditional approval based on clinical benefit), Accelerated approval (based on surrogate endpoints), Approval with Risk Evaluation and Mitigation Strategy (REMS), Approval with restricted distribution or indication limitations, except compassionate use/expanded access programs The following do not constitute qualifying approvals: Approvable letters that require additional actions before approval Tentative approvals pending patent or exclusivity expiration FDA requests for additional information or studies Extension of Prescription Drug User Fee Amendments dates Approval for compassionate use or expanded access programs only Approval only for export or for use outside the United States Emergency Use Authorization (EUA) without full approval Complete Response Letters (CRLs) indicating the application cannot be approved in its current form This market will immediately resolve to "No" if the FDA issues a Complete Response Letter (CRL) or explicitly declines to approve the application. If the drug sponsor withdraws the application before the end of the month, the market will resolve to "No" immediately. If the listed drug is approved before the end of the month, the market will resolve to "Yes," regardless of potential Advisory Committee votes against approval or later withdrawal of approval. Conditional approvals may include post-marketing requirements or commitments and still qualify. The primary resolution source will be official information from the FDA; however, a consensus of credible reporting will also be used.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[     0.8807,     -0.0750,      0.0007]])

Evaluation

Metrics

Triplet

• Datasets: base_eval and test_eval
• Evaluated with TripletEvaluator

Metric	base_eval	test_eval
cosine_accuracy	0.9233	0.9998

Training Details

Training Dataset

Unnamed Dataset

• Size: 21,473 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 7 tokens mean: 12.65 tokens max: 24 tokens	min: 53 tokens mean: 165.55 tokens max: 573 tokens	min: 9 tokens mean: 14.39 tokens max: 26 tokens

• Samples:

  anchor	positive	negative
  Katy Perry confirms relationship with Justin Trudeau publicly	Katy Perry and Justin Trudeau confirmed relationship by August 31? This market will resolve to "Yes" if Katy Perry and Justin Trudeau are confirmed to be in a romantic relationship by August 31, 2025, 11:59 PM ET. Otherwise, this market will resolve to "No". Confirmation must come directly from Katy Perry or Justin Trudeau or their official representative(s), and may come through public statements, social media posts, etc.	Katy Perry Announces New Album Release Date Amid Busy Year
  Jalen Milroe selected with first overall pick in NFL Draft	Will Jalen Milroe be drafted in the First Round? This market will resolve to "Yes" if Jalen Milroe, the QB from Alabama, is selected in the first round of the 2025 NFL Draft scheduled for for April 24, 2025, in Green Bay, Wisconsin. Otherwise, this market will resolve to "No". The resolution source will be the official broadcast of the 2025 NFL Draft.	Expectations Rise for Quarterbacks Entering the 2025 NFL Draft
  Robert F. Kennedy Jr. confirms endorsement of Donald Trump	RFK Jr. endorses Trump before November? This market will resolve to "Yes" if Robert F. Kennedy Jr. announces that he will vote for Donald Trump or formally endorses Trump for President of the United States by October 31, 2024, 11:59 PM ET. Otherwise this market will resolve to "No". The resolution source for this market will be official information from Robert F. Kennedy Jr. or one of his representatives.	Donald Trump Addresses His Campaign Strategies in Latest Rally

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 5,369 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 7 tokens mean: 12.79 tokens max: 27 tokens	min: 51 tokens mean: 166.5 tokens max: 491 tokens	min: 9 tokens mean: 14.36 tokens max: 25 tokens

• Samples:

  anchor	positive	negative
  NOAA confirms average global temperature breaks record	Will August 2021 average global temperature be the highest August temperature on record? This is a market on whether the average global land and ocean surface temperature for August 2021 will be the highest August temperature since global records began in 1880. The resolution source for this market will be the Global Climate Report for August 2021, published by NOAA's National Centers for Environmental Information (https://www.ncdc.noaa.gov/sotc/global/2021). This market will resolve to “Yes” if, for the month of August 2021 averaged as a whole, global land and ocean surface temperature anomaly, as measured by U.S. National Oceanic and Atmospheric Administration, will be greater than 0.98°C (1.76°F) above the 20th century average of 15.6°C (60.1°F), and “No” otherwise. Past data for the month of August can be found here https://www.ncdc.noaa.gov/cag/global/time-series/globe/land_ocean/1/8/1880-2021. This market will resolve when data is first available for the month of August 2021. In ...	Scientists Discuss Long-Term Trends in Global Temperature Variability
  Mavericks overcome Celtics in overtime 23rd	Will the Celtics or the Mavericks win their February 23rd matchup? This is a market on which team will win the February 23rd, 2021 matchup between the Boston Celtics and the Dallas Mavericks. In the event this game is delayed for whatever reason, the resolution of this market will be delayed until the game takes place. In the extraordinarily unlikely event the game is canceled altogether, the market will resolve to 50/50. In the event of overtime, this market will resolve to the eventual winner. Results of this market will be decided by official scores available on https://www.nba.com/.	NBA Analysts Discuss Rising Trends in Team Strategies Across the League
  Lakers win Game 4 against Suns in playoff series	Who will win Suns vs. Lakers: Game 4? This is a market on who will win in the First Round, Game 4, NBA Playoff matchup between the Phoenix Suns and the Los Angeles Lakers, scheduled to take place at 3:30 PM ET May 30, 2021. This market will resolve to “Suns” if the Phoenix Suns win, and “Lakers” if the Los Angeles Lakers win. If the match is postponed to a date on or before June 6, 2021, 3:30 PM ET, the same market conditions will apply. If the match is postponed to a date after June 6, 2021, 3:30 PM ET or cancelled altogether, the market will resolve 50-50. In the event of ambiguity in terms of the market outcome, the market will be resolved in good faith at the sole discretion of the Markets Integrity Committee (MIC).	Phoenix Suns Team Chemistry Highlighted in Postseason Analysis

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• learning_rate: 2e-05
• num_train_epochs: 5
• warmup_ratio: 0.1
• prompts: task: search result | query:

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: task: search result | query:
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	base_eval_cosine_accuracy	test_eval_cosine_accuracy
-1	-1	0.9233	0.9998

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.0.dev0
• PyTorch: 2.8.0+cu126
• Accelerate: 1.11.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}