How to Change Offline samples_per_query

[zixianwang2022]

Hi,

How do I change the values of samples_per_query given to the sut by loadgen?
https://github.com/mlcommons/inference/blob/master/text_to_image/main.py#L421

    def issue_queries(query_samples):
        runner.enqueue(query_samples)

[arjunsuresh]

Hi @zixianwang2022 it's not trivial to change that as it's scenario specific and for each scenario it's value is calculated by loadgen using multiple parameters. In offline scenario, target_qps is the main parameters which determines the number of samples in a query.

If you are running via CM, you can provide --test_query_count=N to the run command to control this.

[NoureddineBouhali]

You can do that by additing the lines in your user.conf :
*.Offline.min_query_count = <desired number of samples>
*.Offline.min_duration = 1
Details for this can be found in loadgen.cpp which controls the number of samples in the query according to the target qps, min_duration and min_query_count in user.conf.