Repo experience survey results

A couple of months ago we ran a survey of our github community to understand our users satisfaction and experience with the mechanics of how the projects are open-sourced with the community. We typically have been looking at the features and satisfaction with our products themselves – this was different because it’s about how the community experiences the project through the lens of the github repos.

Response rate and Repos

We advertised the survey in repos and on our blog and got 191 responses. Most folks arrived from dotnet/announcements (likely from the blog) rather than from sticky posts in repos. Note that we asked respondents to choose one primary repo to answer questions for.

Q1. Which .NET repo do you interact / work with the most? (pick one)

For the rest of the results – only five repos had enough responses to call out the responses on a per repo basis. Before you question if my math is off – the “Grand Total” row includes the results for all the repos including the other repos that are not individually called out.

Satisfaction

The overall satisfaction with repos is good with one exception.

Q3. Overall, how do you rate your experience with the repo you selected?

Q5. How satisfied were you with the responsiveness and engagement of the repo maintainers?

WPF was the main outlier for satisfaction. Drilling into the comments, the main concern was that PRs and issues were not being addressed by the maintainers and there was a lack of clarity on if and when they would be. Internally the WPF team was not sufficiently staffed and did not have the test infrastructure in place to be able to respond to the community contributions. The WPF team appreciates the feedback, and published an updated roadmap and community contribution guidelines to increase transparency with the community back in July. The team is still focused on finishing up .NET 5.0 as per the published roadmap and will share updates on the plan for increasing community engagement on PRs and issues early in the .NET 6.0 release.

Repo Activity

Earlier in the year, we consolidated into a smaller number of repos. We hoped this would make it easier to find the repo they needed. The survey shows that for the most part, people are able to find the correct repo.

Q2. How easy was it to find the repo you needed?

Browsing the code is the most common repo activity. About half of the respondents have filed issues or cloned and tried to build repos.

Q4. Which of the following activities have you done with this repo in the last 90 days?

For those who have filed issues or PRs, the overall satisfaction is high, except for WPF as noted above.

Q6. How satisfied are you with the outcome of the issue or PR(s)?

And for those who did try and build the repo, they were mostly successful, but in many cases it took significant effort.

Q9. If you built the repo locally, were you successful?

Since this survey, we have made incremental efforts to make build easier and better documented. Going forward, we are investigating whether we can take advantage of Github CodeSpaces to help contributors quickly configure a workspace in the cloud. Tooling that is being built to configure the codespaces environment can also be used to configure your local machine.

Working with PRs

In most repos, almost everyone that submitted a PR saw it successfully merged.

Q7. Have you tried to submit a change to the code in the repo you selected above?

They were able to find the right place in the code to make the fix.

Q10. Finding the right place in the code to make the fix

However, we need to make it easier to run tests and debug your changes. This is an area that we will be spending more time upon: for example, in dotnet/runtime we are migrating to vstest to improve the experience in the Visual Studio Test Explorer

Q11. Running tests against the fix

Q12. Ability to debug the code

Inclusion and welcoming-ness

We want .NET to feel welcoming to everybody, and provide a community where people can discuss ideas and problems in a respectful and collaborative way. Thankfully, it seems that most people feel welcome and included.

Q14. Thinking about the community in this repo, How would you rate the friendliness and helpfulness of others in the repo?

To gauge how the .NET community compares to others, we also asked:

Q15. How welcome do you feel in this repo compared to other OSS projects out there?

This shows that there is still work to do. If you have ideas or suggestions for how we help make our communities more inclusive and welcome, please reach out to any of the repo maintainers – we want everyone to feel free to participate or contribute. As always if you witness, or are subjected to behavior contrary to the code of conduct, then please contact conduct@dotnetfoundation.org or opencode@microsoft.com to report any issues.

Freeform comments

When creating the survey, we wanted to make most questions multiple choice as that makes the survey simpler to complete and increases the response rate. We had one question with freeform text to collect additional feedback:

Q16 If we could do one thing to improve your experience in this repo what would it be?

A number of themes stuck out. Some anonymized examples:

Building the source code

• A clear beginner guide to building the source code would be welcome. Even as a 8 year professional dev I had trouble figuring out I needed the latest nightly SDK to run things.
• Make the build process easier
• The build steps are highly complicated and require multiple execution loops until everything build since they can’t be build from VS directly. It also requires knowledge of the ps-scripts that are used to build the system, which is distributed between multiple wiki pages. It’s also not really clear how the “subsystems” interact with each other and why some can be build as Debug and others as Release. The docs seem to explain a lot but seem to be primarily geared towards devs internal to MS or that already have significant experience in those fairly low-level projects. It would be nice to streamline both the instructions and the docs.
• Make it easier to build, test & debug. It’s hard to know which script/command to run, so many options. Should be able to just run dotnet build …
• I would say the most annoying thing about contributing is definitely the fact that you can’t use the standard `dotnet` installed on your device when you build. This has negative impact on integrating with Visual Studio products and also building from the command line It is definitely a bit annoying having to run `build.sh` and having to use the `.dotnet/dotnet.exe` that this installs. Basically, would be great to align the experience using `dotnet` tools to create our own class libraries with how we build dotnet/* projects. The recent strides towards getting `dotnet test` to work is a great example of this.

Lower the barriers to contributing

• I would love it if there was an online remote program that I could participate and I’ll be guided by the team and do some important contribution to the code.
• I really want to contribute, but the good-first-issues are few and far between, and if I’m not watching them constantly they’ll get scooped up. It’s really discouraging when I want to be a part of the community, but I struggle to find something that is at my relative skill level easily.
• I notice that some times new contributors become impatient after submitting a PR, so a way of setting expectations in advance, or just a short message after submitting the PR would be helpful.
• Clarify the lifecycle of an issue, and give better implementation instructions on up-for-grabs issues. Some up-for-grabs mean writing out description etc.
• As someone who has open source experience and .net experience, I think it is hard to do the first steps. I browsed several times through the issues but did not find a good one for beginner. Those tagged with “easy” in dotnet/runtime are mostly very old. I think it would be good, fi there are issues that are really for beginners and whose description make it as easy as possible to understand what is going on.
• Remember that when responding to questions/issues you guys work in the Repo every day and are experts in the area. Most of us asking questions don’t have that level of expertise in the area – hence, the reason for the question. Thus, having patience and answering questions in a robust and well stated way is of most benefit to the community – instead of short, terse answers/responses. Try to better the community by sharing the expertise and have patience with those that are lacking it.

Timely responses to issues and PRs

• Responsiveness from the maintainers of the repo.
• More engagement with issues. A guaranteed triage process that responds within a few days that either puts the ball back in my park with something actionable (e.g. “Here’s what won’t work about your API proposal”), or acknowledges that the ball is in the team’s park and sets an expectation of a aspirational timeframe in which the team will review the issue and provide feedback. E.g. next design review a week from now, after the push for X is done in a month or two, etc. It’s fine to stress that the timeframe is not a promise and is only aspirational, but the main thing is this: Silence is a little demoralizing.
• Sometimes PRs sit for months with no activity, would be great to merge PRs if there’s nothing blocking them
• respond, respond, respond

Understanding the repo structure and changes

• I’ve found it difficult since all the small repos were merged into the big ones that exist now. Lots of online resources have links to the old repos, and it’s much harder searching because you tend to get false positives with common words.
• Since roslyn can be a bit large and intimidating to new contributors, if possible, try to find a way to document the purpose of each project and maybe provide a way to make it easier to find out where to make code changes.
• Merging corefx and corert has made it much harder to follow what is going on at the framework level as opposed to the runtime level. I have limited time to watch the repo, and now a lot of it is spent sorting the wheat from the chaff. While I understand the technical reason for the merger, it really makes things harder for the little guy.
• As a developer interested in the progress made only on the BCL I find it extremely frustrating to sort through BCL and CLR issues/PRs in the dotnet/runtime repo. I would like a clear distinction between those. A label I could filter by would be a good start.
• It would be nicer if the C++ codes for JIT/GC/etc had more comments to help people understand better.

Action Items

Based on the survey feedback we are working on the following to improve the experiences:

• Tooling to make it easier to setup your local or a cloud environment to be able to clone, build, run tests and debug the code from the .NET repos. We will also continue to incrementally improve the experience of building, debugging and testing in the repos.
• Published a roadmap for WPF and working on staffing and tooling to be able to take PRs
• Creating internal dashboards to track the (meaningful) response time to Issues and the time taken to merge community PRs, to ensure they are all being examined and triaged by their respective teams.

We will be running another version of this survey in the fall, to track progress, and include more repos from other parts of the “Developer Division” family.

Thank you to all who took the time to respond. We really do appreciate the feedback. If you have more thoughts and feedback, use the comments below or reach out directly.

The post Repo experience survey results appeared first on .NET Blog.

source https://devblogs.microsoft.com/dotnet/repo-experience-survey-results/