lvyufeng

/

DeepSeek-OCR

like 2

Hello, admire your works! But when I run on Ascend, I encounter some errors like:

• When using mindspore==2.7.0
  • with dtype=mindspore.float16

[MS_ALLOC_CONF]Runtime config:  enable_vmm:True  vmm_align_size:2MB
Some weights of DeepseekOCRForCausalLM were not initialized from the model checkpoint at lvyufeng/DeepSeek-OCR-Community-Latest and are newly initialized: ['model.vision_model.embeddings.position_ids']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
error:
----------------------------------------------------
- Kernel select failed:
----------------------------------------------------
Select CPU operator[Stack] fail! Unsupported data type!
The supported data types are input[Float16], output[Float16]; input[Float32], output[Float32]; input[Float64], output[Float64]; input[Int8], output[Int8]; input[Int16], output[Int16]; input[Int32], output[Int32]; input[Int64], output[Int64]; input[UInt8], output[UInt8]; input[UInt16], output[UInt16]; input[UInt32], output[UInt32]; input[UInt64], output[UInt64]; input[Complex64], output[Complex64]; input[Complex128], output[Complex128]; input[Bool], output[Bool]; , but get input[BFloat16 ] and output[BFloat16 ]
node: @pynative_kernel_graph4000000041:CNode_1{[0]: ValueNode<Primitive> Stack, [1]: @pynative_kernel_graph4000000041:param_Parameter_2}

----------------------------------------------------
- C++ Call Stack: (For framework developers)
----------------------------------------------------
mindspore/ccsrc/plugin/device/cpu/hal/hardware/cpu_device_context.cc:514 SetOperatorInfo

when with dtype=mindspore.float32, errors the same with above.

• When using mindspore==2.7.1:

[MS_ALLOC_CONF] config:  enable_vmm:True  vmm_align_size:2MB
Some weights of DeepseekOCRForCausalLM were not initialized from the model checkpoint at lvyufeng/DeepSeek-OCR-Community-Latest and are newly initialized: ['model.vision_model.embeddings.position_ids']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
error:Invalid sync host to device for tensor storage info in device address:0xfffe8c184b40 device type:1 tensor storage info:0x3db59f40TensorStorageInfo(shape=[1, 257]  strides=[257, 1]  storage_offset=0  ori_shape=[257]  ori_strides=[1]  is_contiguous=1  ori_size=0) size:2056 format:-1 dtype:35 device id:0 device name:CPU shape vector:{1 257 } device point:0xfffe84cf6670 ptr:0x3c3c1d20 from mem pool:0 deleter:1 from persist mem:0 need recycle:0 padding type: and:0x3d72a0c0 device type:2 tensor storage info:0 size:2056 format:-1 dtype:35 device id:0 device name:Ascend shape vector:{1 257 } device point:0x3d1f18d0 ptr:0x12c2cdc6ec00 from mem pool:1 deleter:1 from persist mem:0 need recycle:0 padding type:

----------------------------------------------------
- C++ Call Stack: (For framework developers)
----------------------------------------------------
mindspore/ccsrc/plugin/ascend/res_manager/ascend_res_manager.cc:1159 AsyncHostToDevice

• Environment:

all the same with model card wrote, but mindspore==2.7.0 and mindspore==2.7.1
CANN=8.2.RC2

• code:

try:
    import os
    import mindnlp
    import mindspore as ms
    from transformers import AutoModel, AutoTokenizer
    ms.set_device("Ascend")
    model_name = 'lvyufeng/DeepSeek-OCR-Community-Latest'

    tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
    model = AutoModel.from_pretrained(model_name, dtype=ms.float32, _attn_implementation='sdpa', trust_remote_code=True, use_safetensors=True, device_map='npu:0')
    model = model.eval()

    # prompt = "<image>\nFree OCR. "
    prompt = "<image>\n<|grounding|>Convert the document to markdown. "
    image_file = 'tmp.png'
    output_path = './'

    # infer(self, tokenizer, prompt='', image_file='', output_path = ' ', base_size = 1024, image_size = 640, crop_mode = True, test_compress = False, save_results = False):

    # Tiny: base_size = 512, image_size = 512, crop_mode = False
    # Small: base_size = 640, image_size = 640, crop_mode = False
    # Base: base_size = 1024, image_size = 1024, crop_mode = False
    # Large: base_size = 1280, image_size = 1280, crop_mode = False

    # Gundam: base_size = 1024, image_size = 640, crop_mode = True

    res = model.infer(tokenizer, prompt=prompt, image_file=image_file, output_path = output_path, base_size = 1024, image_size = 640, crop_mode=True, save_results = True, test_compress = True)
except Exception as e:
    print(f"error:{e}")