Skip to content

vllm.v1.worker.ubatch_utils

UBatchSlices module-attribute 

UBatchSlices: TypeAlias = list[UBatchSlice]

UBatchSlice dataclass

Source code in vllm/v1/worker/ubatch_utils.py 
 8
 9
10
11
@dataclass
class UBatchSlice:
    request_slice: slice
    token_slice: slice

request_slice instance-attribute 

request_slice: slice

token_slice instance-attribute 

token_slice: slice

__init__ 

__init__(request_slice: slice, token_slice: slice) -> None

is_second_ubatch_empty 

is_second_ubatch_empty(
    orig_num_tokens_per_ubatch: int,
    padded_num_tokens_per_ubatch: int,
) -> bool
Source code in vllm/v1/worker/ubatch_utils.py 
17
18
19
def is_second_ubatch_empty(orig_num_tokens_per_ubatch: int,
                           padded_num_tokens_per_ubatch: int) -> bool:
    return padded_num_tokens_per_ubatch >= 2 * orig_num_tokens_per_ubatch