o
:/i&
@ sX d Z ddlmZ ddlmZ ddlmZ zddlmZ ddl
m Z W n e
y9 edZedZ edZ Y nw zddlmZ W n e
yT ed Zed
ZY nw dd eeeef d ed
ee dB dee dB dee dB ddf ddZdeeeef dee dee deeeef fddZd eeeef d eddfddZdS )z%Generate plots for benchmark results. )Path)Any)PlaceholderModuleNplotlyexpressio
matplotlibpyplotresults output_pathcolorsitl_thresholdslabelsreturnc
C s |du rg d}|du rddg}|du r;d|d d dd |d d dd
|d d dd d |d d dd g}d
dit t||}d
g| }t| ||}|sYtd dS tj|dddd|d|ig dd}|jdd |jdd |jddd d t
|t | td | dS ) a
Generate an HTML timeline plot from benchmark results.
Args:
results: List of per-request result dictionaries containing:
- start_time: Request start time (seconds)
- ttft: Time to first token (seconds)
- itl: List of inter-token latencies (seconds)
- latency: Total request latency (seconds)
- prompt_len: Number of prompt tokens
- output_tokens: Number of output tokens
output_path: Path where the HTML file will be saved
colors: List of colors for ITL categories (default: green, orange, red, black)
itl_thresholds: ITL thresholds in seconds (default: [1.0, 4.0, 6.0])
labels: Labels for ITL categories (default based on thresholds)
N)z#109618z#FF7F0Ez#D62728g?g?zITL < r i z.0fmsu
ms ≤ ITL < u ITL ≥ TTFTz#636EFAzNo timeline data to plotstartend
request_idtype)
prompt_tokens
output_tokensreq_start_timereq_finish_time
segment_start segment_endduration)x_startx_endycolorcolor_discrete_mapcategory_orders
hover_dataa %{y}
Type: %{fullData.name}
Start: %{customdata[4]}
End: %{customdata[5]}
Duration: %{customdata[6]}
Prompt Tokens: %{customdata[0]}
Output Tokens: %{customdata[1]}
Request Start Time: %{customdata[2]}
Request End Time: %{customdata[3]}
)
hovertemplatereversed) autorangeTime
Request IDT) xaxis_title yaxis_title
showlegendzTimeline plot saved to: ) dictzipconstruct_timeline_dataprintpxtimeline
update_traces update_yaxes
update_layoutpio
write_htmlstr) r
r r r
r
labels_colors labels_order
timeline_datafig r= a/lsinfo/ai/hellotax_ai/llm_service/venv_vllm/lib/python3.10/site-packages/vllm/benchmarks/plot.pygenerate_timeline_plot sH "
r?
requests_datac s dt dtfdd}dt dtf fdd }d}| D ]}|d }|dur.|du s,||k r.|}q|du r5g S g }t| D ]\} }|d }|d
}
|dg } |d } |d d
}
|dd
}|du sl|
du sl| du rmq;|| }||}||
}||}|d| ||d|
|||||| |||
ddd |}|}| D ].}|| }||}|d| |||||
|||||| |||ddd |}|}qq;|S )a
Construct timeline data from request results.
Args:
requests_data: List of per-request result dictionaries
itl_thresholds: ITL thresholds in seconds
labels: Labels for ITL categories
Returns:
List of timeline segments for plotting
sec_timer c S sN t | d }|dk sJ dt | d d }| d }|dd|dd|dS )z'Convert seconds to HH:MM:SS.mmm format.i d z&time seems to last more than 100 hours< 02d:z06.3f)int)rA hmsr= r= r> tostr s
z&construct_timeline_data..tostritlc s0 | d k r
d S | d k rd S d S )z#Categorize ITL based on thresholds.r r r= )rK r
r r= r> itl_type s
z)construct_timeline_data..itl_typeN
start_timettftlatency
prompt_lenr r zReq r z.3frI ) r r r r r r r r r r r )floatr8 get enumerateappend)r@ r
r rJ rN t0requestrO r; irP rK rQ rR r start_time_strttft_end ttft_end_str prev_time
prev_time_str itl_valueitl_end itl_end_strr= rM r> r/ t sz
r/ c C s g }g }g }| D ] }| dd}| dd}|| || ||| q|s0td dS tjdddd\}\\} }
\} } | j|d d
d d d
| d | d | d | j ddd |
j|d dd d d
|
d |
d |
d |
j ddd | j|d dd d d
| d | d | d | j ddd t
t t |}
| j
|
|dd
d d | j
|
||ddd d | d | d | d | | j ddd d t tjt|d!d"d# t| td$| dS )%aC
Generate a matplotlib figure with dataset statistics.
Creates a figure with 4 subplots:
- Top-left: Prompt tokens distribution (histogram)
- Top-right: Output tokens distribution (histogram)
- Bottom-left: Prompt+output tokens distribution (histogram)
- Bottom-right: Stacked bar chart (request_id vs tokens)
Args:
results: List of per-request result dictionaries containing:
- prompt_len: Number of prompt tokens
- output_tokens: Number of output tokens
output_path: Path where the figure will be saved
rR r r z-No data available for dataset statistics plotNrL )
)figsize steelblueblackgffffff?)binsr! edgecoloralphaz
Prompt Tokens FrequencyzPrompt Tokens DistributionTg333333?)rj coralz
Output TokenszOutput Tokens Distributionmediumseagreenz Total Tokens (Prompt + Output)zTotal Tokens Distribution)labelr! rj )bottomrn r! rj r) Tokensz Tokens per Request (Stacked)r )rj axis tight)dpi bbox_inchesz"Dataset statistics plot saved to: )rT rV r0 pltsubplotshist
set_xlabel
set_ylabel set_titlegridlistrangelenbarlegend tight_layoutsavefigr8 close)r
r r r total_tokensrX rR
output_lenr< ax1ax2ax3ax4 request_idsr= r= r> generate_dataset_stats_plot sf
r )NNN)__doc__pathlibr typingr vllm.utils.import_utilsr plotly.expressr r1 plotly.ior r6 ImportError_plotlyplaceholder_attrmatplotlib.pyplotr rv _matplotlibr} r- r8 rS r? r/ r r= r= r= r> sd
[
p