Issues

• Compatibility with transformers > 4.36: error: AttributeError: 'tuple' object has no attribute 'to_legacy_cache'

  BlackSamorez/tensor_parallel#137 · Filter by author Dr-Left opened on Aug 21, 2024
  2

• Customized generate func support?

  BlackSamorez/tensor_parallel#136 · Filter by author MonolithFoundation opened on May 1, 2024

• tensor_parallel int4 LLM is not working since release v2.0.0

  BlackSamorez/tensor_parallel#133 · Filter by author ReinForce-II opened on Jan 2, 2024

• Now, does tensor_parallel no longer support the huggingface trainer?

  BlackSamorez/tensor_parallel#132 · Filter by author HanGyeol-Yoo opened on Dec 31, 2023

• Can I use tensor_parallel to inference for a GPTQ quantized model?

  BlackSamorez/tensor_parallel#131 · Filter by author minlik opened on Nov 15, 2023

• No implement of generate() when using models from hugging face.

  BlackSamorez/tensor_parallel#130 · Filter by author 342215448 opened on Nov 14, 2023

• TensorParallel object has no attribute save_pretrained

  BlackSamorez/tensor_parallel#129 · Filter by author toufunao opened on Nov 3, 2023

• No output when using tensor_parallel

  BlackSamorez/tensor_parallel#128 · Filter by author yyya9 opened on Oct 27, 2023
  1

• AttributeError: object has no attribute 'devices'

  BlackSamorez/tensor_parallel#125 · Filter by author QiueY514 opened on Oct 9, 2023

• ValueError: Model parameters were moved to incorrect devices, did call on model.cuda() or model.to(device)? If so, please avoid doing that

  BlackSamorez/tensor_parallel#124 · Filter by author Khyat opened on Oct 4, 2023

• Max Recursion Error when using with lora

  BlackSamorez/tensor_parallel#122 · Filter by author Ar-Kareem opened on Oct 2, 2023
  2

• RuntimeError: NCCL Error 3: internal error

  BlackSamorez/tensor_parallel#121 · Filter by author smallmocha opened on Sep 12, 2023
  1

• Segmentation fault (core dumped)

  BlackSamorez/tensor_parallel#120 · Filter by author jameswu2014 opened on Sep 11, 2023

• Would it suitable for the multi-GPU parallel inference for llama2?

  BlackSamorez/tensor_parallel#118 · Filter by author aclie opened on Sep 1, 2023

• 2x slowdown using TP

  BlackSamorez/tensor_parallel#117 · Filter by author jph00 opened on Aug 26, 2023

• RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

  BlackSamorez/tensor_parallel#116 · Filter by author SparkJiao opened on Aug 11, 2023

• distributed TP model forward output's requires_grad is False

  BlackSamorez/tensor_parallel#115 · Filter by author lxuechen opened on Aug 10, 2023
  5

• tensor_parallel method distributed=True

  BlackSamorez/tensor_parallel#114 · Filter by author Johnno1011 opened on Aug 2, 2023
  2

• When I try to do the tensor_parallel on NLLB from meta, there is an error

  BlackSamorez/tensor_parallel#105 · Filter by author 342215448 opened on Jul 22, 2023

• Question on custom models

  BlackSamorez/tensor_parallel#88 · Filter by author vince62s opened on Jun 13, 2023
  23

• Does tensor_parallel support data parallel and tensor parallel hybrid training?

  BlackSamorez/tensor_parallel#85 · Filter by author liguodongiot opened on Jun 7, 2023

• Does tensor_parallel support multi-node tensor parallel training?

  BlackSamorez/tensor_parallel#84 · Filter by author liguodongiot opened on Jun 7, 2023
  6

• Can I parallelize just one large layer?

  BlackSamorez/tensor_parallel#83 · Filter by author chinmayjog13 opened on Jun 6, 2023
  1

• Great work！ and can this work with deepspeedzero?

  BlackSamorez/tensor_parallel#75 · Filter by author OpenSaozi opened on May 22, 2023

• Slow inference performance for large Llama models compared to naive MP

  BlackSamorez/tensor_parallel#66 · Filter by author sgsdxzy opened on Apr 6, 2023
  26